RMT elements drain our resources with incessant account hacking. This is an Industry-wide problem and most MMOs are beset by non-stop issues with players’ assets and hard work being stolen and sold off for real money by organized criminal networks, who will stop at nothing in order to profit.

Until last summer we experienced regular issues with mass hacking a couple of times a year, usually around Christmas and then again in the middle of summer. Nowadays, this is pretty much constant with hundreds of accounts being targeted every day. Many of those are old trial accounts or accounts that have been disabled for years and don’t hold anything of value to the RMT types. Unfortunately, there are still a lot of accounts that do have assets and ISK that end up being cleaned out – even the characters themselves being sold off. The damages are sadly not repairable in some cases, regardless of valiant efforts by GameMasters to help the unfortunate victims. The cost in resources is high for Customer Support, with highly trained and experienced GMs working almost exclusively on hacking cases – good people whose time and talent would be much better spent on enhancing the gaming experience and increasing the quality of service we are able to give to our customers.

The methods the hackers use differ and constantly change but the result is always the same – your account is ruined. However, users can take steps to limit the chances of getting attacked and following is a list of things that can help make your accounts more secure.

Do not use the same usernames and passwords for different games
Every day, we see countless attempts to log in with pairs of usernames and passwords, amongst them many usernames that don’t exist in our systems. Obviously, those non-existing usernames have been harvested from somewhere else. They have been gathered via keyloggers, phishing sites, trojans, hacked forums and whatnot and long lists of such username/password pairs are traded between RMT types for use against gamer accounts all over the place. A good way to avoid problems with this is to simply use different login details for each game.

The same should also go for third-party sites and forums as those are quite often targeted by the hackers to harvest login details. Login details for such third-party sites may or may not be encrypted so keeping separate usernames and passwords for your gaming accounts is the way to go.

Change your passwords regularly
If your login details have been harvested, a regular change of passwords may prevent attacks from being successful.

Use strong passwords
Passwords should be complex and difficult to guess. Using a mix of numbers and small/capital letters can reduce the dangers from brute-forcing and lucky guesswork on part of the hackers. Avoid using common dictionary words and keep in mind that longer passwords are less vulnerable than short ones. A minimum length of 16 characters with a mix of lower case, capitals and numbers is strongly recommended for heightened security.

Do not share your login details with anyone
If you give someone your login details, your security is only as good as his. If he is hacked, you are hacked – given that he won’t simply use or sell your details himself.

Don’t accept files from sources you don’t know
A lot of the mal-ware on the Internet specifically targets gamer accounts. RMT in online gaming is a huge racket – your login details are a valuable commodity and the pitfalls are many. Keyloggers and trojans – all geared towards the destruction of your accounts lay in wait, poised to strike when you open that file or go to that website. Phising schemes abound and social engineering is rife, on an Internet that often seems without law or consequence. One cannot be too careful – it’s not paranoia when they’re really out to get you.

Regularly scan your systems for security threats with up-to-date anti-virus software
Protect yourself by running updated anti-virus software to find and fix security threats that may have found their way onto your systems. There are many such programs available, some free and some not free, but definitely worth spending time to set up and the money to purchase. It’s imperative to maintain a virus scanner and Operating System by actively checking for new updates and applying them, especially for the virus scanner. Using a firewall is also recommended as an optional measure.

Also see this forum discussion for more suggestions on better security.

By following the simple steps above you can make your accounts more secure and limit the dangers of being attacked by hackers who are after your stuff. Please be sure that we are not sitting idly by either – we are currently working hard on account security upgrades to get this problem under control. There are several items on the menu and the we hope to implement the first countermeasures in the next few weeks. However, we urge all of you to step up your own security at home by following the suggestions listed above.

Together we will vanquish this evil monster!

A small army of developers from CCP will be at GDC San Francisco (Game Developer’s Conference), March 9-13. Specifically booth #2505 in the Career Pavilion in the South Hall. As you can imagine, we’ll be doing some heavy recruiting, some interviews and lots of note-taking. We’ll also be talking shop on Tyrannis and would love to chat with any EVE fans who make the trip.

Powerpoints are prepared and black dev shirts have been ironed. Come join us at any of the four speaking sessions we’ll be a part of–from fighting the good fight against RMT to cloth simulation and from scaling EVE’s universe to scaling CCP itself.

In short, the answer to the question is nothing less than “The CSM4 summit in Iceland has been the best summit so far.” The CSM1 summit was also very good, but for a different reason. Back then we were going in blind and much of the summit, although nowhere stated specifically, was about testing the theory and establishing work methods and traditions. The CSM2 summit was mostly about polishing and getting the small nuisances out of the way when it came to having the CSM and the CCP representatives talking together. After the CSM3 summit it was clear that further enhancements had to be done to the CSM as a whole, so we made several more steps there.

So, how was the fourth CSM summit different? Several key things were changed. First of all we had two days of CCP discussing current and future design ideas while we had only one for CSM3 and none for the first two. And not just discussing but brainstorming with the CSM. And not only that, because by having the CSM in Iceland so early in the development process of Tyrannis a subtle effect is noticeable in the overall design course EVE is on. Taking a conscious step backwards and have the CSM discuss macro designs and the general heading of EVE’s development was taken.

Another change is that we had a dedicated secretary taking the minutes, instead of having the Secretary of the CSM taking them. That resulted in the minutes being published six days after the CSM was in Iceland! The minutes are available here. And it left the CSM secretary free to participate in the discussions as keeping up with 11 to 13 people is doable but leaves little room for the person to express itself.

As the minutes indicate there is solid progress going on with the CSM and you should all keep an eye on this space over the next weeks when we will give you further insight into it. Some quantitative data will be given out at a later date and the changes being made to the CSM will also be detailed.

But what did the CSM members feel about the summit?

Tim, aka Sokratesz commented that ”I was very skeptical initially, but the trip has convinced me that CCP has an active interest in the playerbase and is willing to listen. One of the best moments for me was when Noah [CCP Hammer] grabbed a pen to take notes during a heated debate between council members.” He also noted that ”The goons weren’t living up to their reputation. They were actually being useful”

Asher, aka Mrs Trzzbk states that “Making the CSM an official stakeholder in the development process was extremely significant. It will be in a much better position to address the long standing concerns of the player base while still remaining a valuable sounding board for CCP. From this, to our discussion with the GM department, to the sincere debates with CCP staff on features both large and small I believe we had an extremely productive summit.”

Jason, aka TeaDaze: “I wasn’t sure what to expect from the summit other than further debating the issues we’d already raised with CCP. I was pleasantly surprised that whilst we did spend over a full day on those issues that a large amount of the time was spent on discussion and feedback about current and future plans. I can see improvements in the CSM process but going forward it needs to gather even more support from the player base. If the stakeholder promise works out I think the CSM is in a good position to achieve something unique in the gaming world.”

John, aka Zastrow, the only reelected member of the CSM has the word: “While there has been very little communication between CCP and the CSM between summits, at the summit it is very clear that CCP takes the CSM very seriously. For 3 days straight we sit down for all day meetings with the Executive Producer, Lead Game Designer, and a senior programmer and go over their proposals for the next expansion and our summit agenda.”

Until the next blog!

March 3, 2010 marked the final day of the collection of PLEX to aid victims of the devastating Haiti earthquake. EVE Online players were extremely generous: 2776 PLEX were donated through 1107 contracts. An additional 9.5 billion ISK was donated, enough to purchase 37 additional PLEX from the market. 100 percent of your donations will be given to the Red Cross. Thanks again to everyone who participated.

Since November 25th 2009, a few days before the deployment of Dominion, we have experienced frequent unscheduled reboots of Tranquility. Almost all of those were due to a bug in the networking subsystem that causes the SQL Server to fail over.

Why, oh, why and what are you doing about it?

As soon as the first failover happened, we, per our policy, opened up a support case with the vendor regarding the incident, since our logs, surprisingly, showed nothing. Their response was that the problem had been caused by a race condition in the system.

We have worked closely with the vendor’s support and development teams in an attempt to isolate the bug, collected vast amounts of diagnostic data and implemented changes that were considered potential solutions by the vendor. We believe we’ve found a workaround that makes it unlikely that the bug is triggered, but does not 100% prevent it. This has yet to be confirmed however.

As one can imagine it is difficult to diagnose a running, high performance production environment like ours without causing lag or other performance or reliability problems. The vendor has been working diligently to attempt to reproduce this issue in their lab, although collecting diagnostic data from similar systems presents a major challenge – doing so without negatively impacting performance levels for customers.

We do have programmers and virtual world system administrators working on putting together a test script to run on the database server we use for Singularity and Multiplicity, and if we are able to reproduce the issue there, we can supply our vendor with code that reproduces the problem in their lab.

I, personally, have been spending quite a large part of my work hours the last 3 months communicating directly with the vendor, collecting diagnostics data, setting up collection tools and working on things related to solving the SQL Server issues.

In short, we are using all the resources at our disposal to resolve this issue. It is a high priority issue for all parties involved as it affects not only our system and customers, but can affect equally massive systems and user bases using similar network and database solutions.

What have we done already? What do we know?

We know that problem lies in the TCP stack and likely has something to do with handling of closed or closing sockets. Our vendor has asked us to implement a few potential fixes or workarounds. We’ve adjusted various networking features and upgraded our SQL Server engine with a version that has a workaround for issues of this nature. The database handler in the EVE application server uses session pooling and we’ve experimented with changing various settings there. Turning off recycling of idle sessions seems promising as a workaround that makes triggering the bug less likely.

We still are working toward a fix, as I said before, and we seem to be able to make the failovers happen less frequently with the latest workaround. Expect to hear more in the near future on our progress with this issue.