Wednesday, January 11, 2012

Star Wars: The Old Republic - Performance and FPS Issues

This is a follow up to my previous blog at http://mallen22842.blogspot.com/2012/01/customer-service-and-star-wars-old.html.

Very recently BioWare's game director, James Ohlen, told Eurogamer that "most of our players aren't really having performance concerns".

Now, so far I have been very impressed with the game's class story lines (they are nothing short of awesome) and BioWare's attempt to stick to the lore of the Star Wars universe.  The game in general feels very much like a clone of World of Warcraft (talent trees, harvesting nodes for crafting materials, skill-click combat system, a horde [empire] and alliance [republic] faction system) with a few minuscule differences. This is not necessarily a bad thing because it is something the community is familiar with and... let's be honest, it works for MMOs.

But, something about the way BioWare has handled their customer service and the games overall performance has rubbed me the wrong way. To a point where I have lately been on the fence about  setting up a subscription and/or waiting for the dust to clear.

Mr. Ohlen claims that the players are not really having performance issues yet my own testing on both of my rigs, a "Horrid FPS" thread that seems endless, and the following images speak otherwise:




Those images alone contradict Mr. Ohlen's claims that there is an insignificant FPS and/or performance issue.  This easily ranks as the #1 community driven issue with the game at the moment. My assumption of this is simply by a visual inspection of the forum posts and reviews left on various websites marketing the product. This FPS and performance issue even appears to be superseding game crash issues.

This wasn't enough for me to base an opinion on though so I began researching the game a little further. What I found was interesting if not informative.

An article was written by HeroEngine on November 28th, 2011. This article detailed some very interesting facts about BioWare's initial contact with the game engine developers.



The key here is the statement "There are whole sections of code that is only roughed in and not optimized for performance or security.". That is right, not optimized for performance. Interesting... Now, BioWare claims that they had tons of engineers finish things up themselves. But did they actually hurdle the performance issues so clearly pointed out by the engine creator? Or, did they miss some fundamental piece of the equation?

The details given to the actual community leading up to the article at Eurogamer leave a contradictory taste in the community's mouth.  See some of the community messages sent out by BioWare below (which have been very obscure and have a canned feeling to them):






A couple of highlights to take note of. "The FPS issue is so varied, and any number of factors could affect it". But, you have already stated it is due to low-end computers and there really isn't a performance issue. And the answer relayed to me via Mr. Ohlen's public release is clear, upgrade your gaming rigs. It can't be a varied issue but, also not an issue.

And it should be noted that I believe the gaming community has a difference of opinion regarding what systems are low-end and not. BioWare should know better that if the computer meets your minimum and recommended specifications and still has an issue, that the problem is not necessarily the user's computer.

Another highlight. "I appreciate the frustration... I assure you it's being looked at." Again, it is very confusing as to what is being looked at if the implication is that the problem lies with the customer's computer. A contradictory statement.

Another highlight. "...based on the amount of forum posts and CS ticket volume in relation to the total player base population, most people aren't experiencing performance issues.". I implore you to review the masses of forum posts in the various screen shots above.

Mr. Ohlen implied that 5% of players are having this issue. Based on Baird Equity Research and an article posted by Massively by Joystiq on January 3rd, 2012 the game currently has roughly 350,000 concurrent users. 5% of 350,000 concurrent users is roughly 17,500.

Let's assume these users purchased the cheapest edition of the game (Standard Edition $59.99 US). We are looking at potentially around $1,050,000 US dollars in profit for a game that can't be played. Specifically what was expected of the product and what was received. And if those users cancelled their monthly subscription ($14.99 US), a loss of roughly $250,000 US dollars a month.

To me it is pretty amazing that this issue has been handled the way it has. I personally hope that BioWare gets the FPS optimized properly (see this article by HeroEngine about controlling FPS). And I hope that they can clean up their act when it comes to servicing their customers properly.

As always, this is simply my opinion and mileage may vary.

Monday, January 9, 2012

Customer Service and Star Wars: The Old Republic

This morning I wanted to share a Customer Service experience that I recently had with the game development company, BioWare. First a little background information, then the problem, and then I will finish by discussing my Customer Service experience.

I recently pre-ordered and purchased their new MMO (massive multi-player online) game called "Star Wars: The Old Republic". I currently have a subscription that is good until January 19th, 2012. At which point the expectation is to start paying for a monthly fee. I personally subscribed for the six month at a time block. Which is to say, I pay one time and am good for six months of play. As opposed to per month.

Getting back on track... The game released and wasn't exactly clear of software bugs. One, of a particular note, is known as the "Horrid FPS" bug. The games frame rate drops to dramatically low numbers but, only does this in particular areas of the game. Or, when there is a heavy presence of other characters nearby.

Some other people would have you believe that the issue is strictly on the user's computer. "You have a poor machine and it can't handle the game." they might say.  They would also be inclined to throw numbers at you and say "One million people are playing just fine, who cares about the five percent having this problem?".

First, in response to that I would have to say that the problem is not so easily explained away with a simple computer upgrade. Nor can these people give us reports on who has the problem and who does not. For they themselves do not work at BioWare nor have access to BioWare's reports.

I decided that the best course of action was to troubleshoot my own computers (a laptop and a desktop). Both are equally suited for playing the game. Easily surpassing the games relatively low minimum specifications.

Some steps I took are:

  1. Updated my drivers on both computers to the most current drivers available (this currently happens to be AMD Catalyst 12.1a Preview drivers).
  2. Ensured that the computers were up-to-date on all relative pieces of software (operating system, DirectX, etc).
  3. Tweaked the game's client configuration files. In an attempt to lower settings.
  4. Modified in-game settings up and down. Just to see if I could get FPS gain from the changes.
  5. Utilized the Catalyst software to force certain graphic settings (Anti-aliasing, etc). Again for FPS gain.
  6. Attempted to run the swtor.exe application using different compatibility settings. Such as running the game in Windows XP SP3 mode.
  7. Ensured game is set as being ran as an administrator.
  8. Using Windows services, practically turned down all other applications and settings. To maximize my machine's performance.
  9. Through hardware upgrades, increased my machine's RAM (planned on doing this anyway).
  10. Overclocked my video card on the desktop machine (do this at your own risk).
  11. Installed gaming boost software, such as Rivatuner.
  12. Ensured game was forced to run in 3D mode and never drop into 2D mode. To help prevent FPS stutter.
I am sure I am missing some steps I have taken but, needless to say I extensively applied my technical skills to fixing this frame rate issue. But, alas, I was unable to correct the low frame rates. Which was surprising when games such as Neverwinter Nights 2, Guild Wars, World of Warcraft, Call of Duty, etc could all be played with zero frame rate issues. On the highest graphical settings. The graphics in Star Wars: The Old Republic were less then exemplary in comparison to these others.

I then began to monitor my resource usage. And this is when I stumbled upon the problem at its core. I noticed that during frame rate highs (70-100 FPS) the game was utilizing about 75% of my processor, about 50-75% of my RAM, and about 80% of my video card's resources. But, when dropping to frame rate lows (0-5 FPS) the game stayed exactly the same with the exception of my video card resources. The game was only utilizing 20% of my video card.

So this immediately told me what I needed to know. BioWare was not communicating (via my drivers) the correct amount of resources in needed to load upon my video card. And I decided it was time to start looking into reaching out to BioWare's support team.

A spokesperson for BioWare stated the following on January 5th, 2012 at this thread.
Because this FPS issue is so varied, and any number of factors could affect it, it's taking a bit more time to investigate. At the moment, our developers are going over logs and are working on a more detailed blog post that we should be posting soon. Thanks again for your patience!
Which has yet to be blogged at the time of my writing this. I was able to jostle a BioWare spokesperson myself, with the following response on the forums (yes, I hijacked the thread a little bit):
FPS / framerate performance issues are also being looked at. Like the 'Ability Delay' issue, framerate issues are not something that can be solved in a quick and easy way with a single fix. Investigation into the various causes of framerate issues takes time, and solutions may not immediately work for all. Work continues to optimize and improve the game.
Which I immediately replied with:
Look at it from my perspective though. I truly want to subscribe to the 6 month subscription. Which I have (so far). But, if I am being told my FPS issue could take an extended amount of time research to address, why keep my subscription which takes affect January 19th, 2012?
And was told:
Feel free to share your dxdiag and machine specs in this thread - that's where we're collating feedback and having the development team actively monitor to help improve things for players. I assure you it's being looked at. 
My frustration was really high at this point. I did not want a stock / canned response. This took me to the next level. I needed to call someone. So I contacted BioWare on January 6th, 2012 at 1 (855) 467-9867. After navigating the various prompts I reached a person, relatively quickly, calling themselves Mel. This appeared to be an overseas (India?) call center based on the background noise and representatives accent.

Mel asked me for some basic account information before he began discussing my issues with me. He asked if my computer met the minimum game requirements (easily) and if I had tried adjusting the in-game settings (first thing I did). I then detailed other various troubleshooting I had performed on my own. And told him that I pinpointed the issue was the game engine communicating with my drivers.

Mel placed me on hold to go see if there was anything else he could walk me through over the phone. After about five minutes, he returned. Mel told me that he would have to have me email the details of the phone call to him (shouldn't you be writing this down in your call log Mel?) and that he would escalate to the next tier of their support team.

I informed Mel that I did not wish to send in an email and would prefer speaking with a member of this support team myself. I explained to him that I am a software engineer and could easily relate to them the details behind the problem. It was at this point he informed me that he could not transfer me because it was not allowed.

I said "Mel, I know very well that you or your supervisor can transfer my call to the other department. Please put your supervisor on the phone.". With which he told me "Well, let me see if I am allowed to transfer you." and placed me on hold yet again.

It was at this point that I was "allowed" to be transferred to the "Technical Support department". Which I can only guess what department it was that I was currently speaking too.  Now, buckle your seat belts because this is where things get interesting.
The mysterious representative: "Is there something I can help you with?"
Me: "Yes, I am sorry... I didn't catch your name."
The mysterious representative: "(mumble mumble) Can I help you?"
Me: "I'm sorry, I still didn't catch that. Can I have your name please?"
The mysterious representative: "Sir, we don't have names here. Is there something I can help you with."
Me: "Excuse me, you don't have names?"
The mysterious representative: "That's right."
Me: "Okay, I would like your operator number or to speak with your Supervisor please."
*click*
Wow is all I could say (actually I said some other choice phrases out loud). I immediately called back and this time got a Lawrence and what sounded like a completely different call center.

I relayed the entire story to him and he sounded genuinely astonished at the experience thus far. It was Lawrence who provided me with the details that "Yes, there is something incorrect on our side we are trying to address" and "We are having trouble pinpointing the exact problem so it is taking longer then we had hoped to correct".  Lawrence was very apologetic and I told him I would be more then happy to detail the problems via email. As long as he assured me it reached the development team. Lawrence assured me it would.

I then sent a very detailed email to their team via reply to a generated email to myself (thanks Lawrence).

And to this day, I have not heard back.

All in all, this is one of the rougher game start ups I have seen. And the Customer Service is a bit lacking, to say the least. My recommendation to anyone considering buying this game is to wait. And to read the game's forums (you can do this publicly) until some of the dust clears.

While an excellent game, some of these finer points are actually hurting the game. I will post a follow up when I have something to follow up with.