Saturday, October 6, 2012

Eve Vegas: CCP Stillman Discusses Bots and Security

Edit: CCP Sreegs was kind enough to contact the Nosy Gamer and give a deeper, clearer explanation of the goal towards dealing with RMT

I wandered off to Nathans to have lunch. Real hot dogs are hard to come by in my area. Good stuff. Some people were late to filter in. When I returned people were still getting badges and goody bags. The goody bags had Eve centric poker chips in them. Very cute souvenirs.

The reception was detailed out a little bit. I wound up not going and chose instead to hang out with my corporation. My first goal was to go but the rules of it made it not that appealing to a nondrinker with no one to socialize with. It did sound hysterical.

The way the set up was described was that Jita would be in the middle. There would be girls in spacesuits serving drinks. The tables would be around in a big center and players were to start in their home systems. Then to change tables they would have to take Sov through drinking games.

From Eve Vegas 2012

Hong was sitting next to me. He had teased me when I unpacked my bag with laptop, hot spot, power and secondary battery backup. Then he stopped and leaned back saying, "I wish I had something to write some of this stuff down with." I handed him a pen and a pad of paper from my bag. He then managed to write about two pages of notes for his own blogging.

The player discussions I will lump together into a different blog set from the Devs for (hopefully) easier browsing. The notes I took for them are more impressions due to the different flow of information.

CCP Stillman: Security and Bots

Past, Present and Future of Botting and RMT. (When I saw this title I had a little vision of the Nosy Gamer spinning around in his chair cheering)

When CCP first attacked the problem they went back and were able to track about when bots started appearing in Eve. They found that the first bot was released around Exodus in 2005. And then the trend went up by the graph displayed. (This was a very graph heavy presentation)

Unholy Rage: This was CCP's first irst attempt to solve the botting problems. Botting and RMT has become socially acceptable because of the inaction on the part of CCP. The introduction of PLEX also set the stage for an alternative to RMT. Then there had to be Company wide understanding of the issue at hand. Also, a company wide effort. Until this point, no one had paid attention or really cared. Thusly, Plex was a direct step to try to combat the RMT problem and give CCP some control over it.

The plan: They had everyone pitched in to try and contain the issue. Research and statics provided intelligence to GMS. A handful of GMs were tasked with verification and laying down justice. It was a large, time consuming project.

RMT temp bans from 2009-2010 were around 9k a month. Once they did the temp ban and rechecked they would issue a permanent ban the next month. This would cause the next month to have around 10k bans to it. As these waves of bans went into place they saw the RMT market dropped. The bans also dropped the ISK flowing into the game by huge amounts (more graphs). This is a graph that shows unholy rage did a big dent in the ISK coming in. Another side effect was that a noticeable server load decreased with the bots being banned. Due to their 24/3 nature of constant activitiy they put a much larger strain on the game that a real player did making another aspect of the bots one in which they starved real players for physical game resources.

The bad thing was that Unholy rage was not sustainable. They couldn't keep it up long term. It was inefficient and heavy on man power. The people tasked to help with it had other developmental duties. To continue to combat the bots and RMTrs They needed a more specialized effort. (quoting charts and power point).

In 2010 they hired CCP Sreegs. In March 2011 he put together the task force. CCP Sreegs, CCP Chronotis, CCP Stillman,CCP Masterplan, CCP Grimmi and CCP Pollux were combined. They made a cross functional team that had someone from each department to work on stuff in their free time. Their primary focus is enforcing the EULA. CCP Sreegs and CCP Pollux are full time at team security and everyone else is about 10%.

CCP Masterplan and CCP Stillman began to work on a prototype program to find bots. They targeted specific bots. They then went one step at a time to prove that they could do it reliably. They were successful at shutting down the bot that they targeted.

They also started the small ban waves instead of the massive ban sprees. What wound up happening is every 7-14 days they did a ban wave with their three strike cases. The third strike also takes out every account that the person has not just the botting accounts. They said it had a big impact. But, that was not sustainable as well and Incarnia's development time and eventually fall out sucked up all of their time.

When CCP Unifex came on scene there was another reorganization. This one led to Team Security as we know it now. A team focused on security. They made sure to Consolidate responsibility to prevent duplication of effort. There had been to many unsustainable projects. CCP Sreegs runs the team. CCP Stillman joined as an application security analyst. CCP Arkanon and Peligro are the IA part of the team.

New Goals:
----Better able to deliver internal tools.
----Work together with other teams to deliver new features.
----Ability to focus and evote chunks of time to project. This also let them focus their time on doing this project and not get dragged off by others.

Now that they were back on track they rewrote the systems conceived by the Eve security taskforce. They made huge improvements to detection rate. A ton of nobs and switches were added. It was now easy to improve on. Their focus was on slow burn. They wanted to have less of an unholy rage type thing and more of banning small groups of people on the regular.

The data collected is not 100% consistent. Collected across a week. There is inherent noise in the system based off of how they do things.

Now time for graphs.

Since team security has started banning people again they do bans every 24 hours. There is a spike of bans at first and then the graph levels off. Every couple of weeks they are detecting a couple hundred people. He said its a good level to be at from the thousands they used to find.

They detect market bots (some people don't believe it, but they do, he says) mining bots, mission bots and ratting bots as well. There are also courier bots that do the courier missions. Market bots are 2% of the people they ban. He says the data does not show a rampant market bot problem. He says the individual market bots have a per account larger impact then many other types of bot. Mining and Ratting bots are the two biggest spheres on the graph. Third place was mission bots. The other types created a very small slice of the pie.

A chart of Seized ISK this summer. They've calculated a removal of 3 tril from the economy. It is not just a flat process. They calculate how much a person as made from their botting as part of the removal of the ISK. It is something they calculate daily.


Slow burn -
---Gradual changes.
---Watch numbers and impact to assess success or failure.
----------Examples: Character transfer blocks.
----------Size ISK automatically.
---Other bank end changes they can't discuss. They make changes on a regular basis to change up their detection to avoid becoming to easy to avoid by the bot writers.

Band things on characters. RMT is a perm ban as soon as detected. Character's isk is 3 trillion isk. But assets is about 11 trillion. Corporation isk is about 909bil and Corporation assets of 1.3tril. These are the bot corps.

Skillpoints: Generally the people they are banning are people with low skill points. So most of the bans are new characters. Because they also ban all accounts they are not seeing many bans across older accounts. That risk of losing that high skill point account seems to be greater then many are willing to take.

Ships types flying at the time: Lots of tengus and macks are banned. Hulks, Navy Issue Ravens, lots of iterons from macro hauling missions. An interesting thing he noticed is that Nyx are represented. 32 of those baned. 8 aions, 3 avatars, and lots of other small grops of random ships.

He said that there are a few alliances that are big into botting but most are not. The graph here started at a very high number (of which I do not remember) and then plummets down to a very level wiggly line along the bottom of the graph representing everyone else from the 'big into botting' group. No group is named.

The average botting income was around 200 billion when this started. But as they have improved their detection the average income has gone down to more like 50 billion an account due to their inability to bot 24/7. Per account they (the botters) are making less income. They have to make more accounts and spread that botting out across them to make as much isk as they once did. This is making them easier to detect and forces them to spend more money and ISK to break the rules.

As the time is going on the ISK entering the economy from botters has decreased greatly. The ore mining chart also shows a decrease from 1k veldspar to hovering around 300. (I think this was units mined or owned? I'm not sure) Some of the graphics for the higher end ores were very busy but he said they were showing a decrease from the bots across the board.

The future:
---Continue the slow burn.
---Continue to improve the anti-bot measures.
---Improved tooling for Team Security.

Then a video about POSs and RMTs.

A lot of POSs were blown up. Every single thing inside of them exploded. No drops, just destruction. The bubble goes down the everything pops. The room sat, quietly, and watched explosion after explosion and it was explained that these POSs had harbored RMT. When the video finished it was very quiet.

They will go after alliance affiliated operations if they have must too get bots. Now is the time to start this phase and this is their response. The people who owned the POS were not RMT but let RMT people use the POS.

No more social acceptance of botting.

They said at fanfest they were going to look at it. Now they have dealt with the low hanging fruit and now they are taking the next step. Next they are going to go after alliances and corporations hosting botting and RMT. They are no longer going to allow that. They will make sure the alliance feels the pain from harboring these EULA breakers.

However: He said they are not going to do anything like banning the entire alliance and walking away. They are going to reach out and contact people. This to will be a slow burn. They are going to take small steps and see what happens like they have with botting. They won't kill the entire alliance (at least not initially). Communication is going to be a key goal here to get alliances to clean up their ranks.

Now for Q&A: (People leapt up to come ask questions)

Q---The reporting mechanism. has CCP thought of giving a bounty for reporting bots? Can we get some isk if we report them?
A---They don't want a feed back mechanism in game to their detection. As soon as they offer bounties they are leaking too much information. Now their detection system has an ability to be tracked. While they want to reward people they have another mechanism for that. If you know a big bot/rmt op is going on mail security@ccp and you will get 2 plex if it turns out to be good intel.

Q---How are they going to deal with rental alliances that have no impact on what is going on?
A---If at the end of the day if you are going to let people enter your alliance and you are receiving rent money from them you are on the hook as well. You may not know what is going on but he thinks that people need to do a bit more background.
Q Continued --- But that switches enforcing what the renters are doing from CCP to the player but CCP is paid to take care of it and now making the player do their job.
A---He doesn't want to punish people for it. But there is word on the street that high end alliance leaders are involved and they are going to be affected.
Q Continued ---If you have proof then kick them but don't make us do your job
A---We are not going to kill the alliance over night. We will talk and find out and seek. Its still up in the air as to how it will be handled but nothing is set in stone.

Q---Earlier it was said that the team has had several compositions before but they have died because of lack of manpower and teams and stuff. How does he feel about the current possible longevity of this time? From a player perspective it feels that CCP is not as serious as they say they are. What are you doing to make us happy?
A--- He says that there is a lot to consider. CCP has seen that they are helping because they also solve other things like hacking problems and sever problems and such. That the slow burn is not as fast as people like as it gets deeper and burns in more the issue will snow ball and players will see a larger change. As they can address small problems better and reduce the background noise they will be more efficient.

Q--- The plans to hold alliance responsible. When can they get CCPs tools to locate botters in their own alliances?
A---CCP ain't giving out their tools. Once they put them out there, out of their control they have their their power. The people will have them to learn how to get around it
Q Continued--- But its not right to hold the alliance responsible for what they can't detect
A---If they think that the alliance is not clearly doing something wrong but its a focused problem they will reach out to you.
Q Continued---Will we get account names of the botters then?
A--- They want to work with the players and they will look into it. But no scarlet letter. Also this isn't going to be an overnight thing where they just blap the alliance out of space.

Q---Are they changing the EULA for alliance leaders that they have to be proactive to avoid the bans? It sounds like we are not getting account info, tools, and CCP expects players to give them everything or they are gone! It seems unreasonable that alliance leaders are going to be on the hook for this corp coming in that is botting and no one even knows. The question is how does this fit into the EULA to make the alliance leaders responsible for botting on other players accounts?
A---Right now EULA says that if you knowingly interact with botters you are responsible.
Q Continued---How is the alliance leadership affected?
A---This isn't going to be instant.
Q Continued ---What is the time frame? What safeguards do alliances have from their bad seeds?
A---If CCP thinks something is up they are going to reach out.
Q Continued-- Doesn't that violate the EULA that CCP is giving the personal info?
A---They are not going to violate the EULA but they will reach out to people
Q Continued ---Sounds like there needs to be more thinking. This just sounds bad on so many levels and CCP needs to think about what they are doing (Stillman is handling this well. He continues to say that this is going to be slow and they are going to work on it and they have not built any policies yet.)

Q---Many other MMOs have similar problems but its often account hijacking. Does CCP have account hacking problems?
A---Stillman says that yes its also account hacking and other problems from the big RMT sites and they find out about that stuff as well as they work through their detection process.

Q--- Has CCP considered embracing it with a flag and penalties if people want to bot? To accept it. It happens!
A--- Playershave the PLEX that people can use. They can buy game time. That is the embracement CCP has had. No botting is okay.

Q---What sort of response would CCP be looking for from the alliance leaders?
A---They are not overly concerned about the alliances at the bottom of the issue. He will start with the alliances that are obviously botting and he can't help but know that they know that there is botting. They will start with the biggest impact first. If they think it is unknowingly happening behind peoples back they are not just going to ban the innocent. They do want it addressed.

Q---Account security? Two factor authentication?
A---There is a lot of backend work to this. The way that it will work is that the new system they are working is tied into the new third party functionality. They have to roll that out everywhere before they do the two part auth. Also they have to figure out distributions. They are working on it. Answer is 'soon' but they don't have it all in place

Q---How about letting Eve handle Eve. Considered a script to allow PvPrs to go after bots and let us self police without suicide ganking or chasing down ratting ravens? Players will kill them if CCP lets us! A---They have discussed the player enforcement aspect but they feel it is more of a game design issue and something for them to address.

Q---It's true that every alliance bots to some degree, right?
(I found that to be a very broad and surprising answer)

Q---Is there anything proactive that has been considered? There has been very little feedback to the community to show people that botting is bad? To teach people its wrong before they decide to do it? A---Um.... (long pause) we do have plans to do more proactive work and there is a technical perspective to it and social acceptance to it. They want to teach people that it is wrong but they have not been addressing that side as much.


  1. It would be funny if CCP could track down the alliance leaders who advertise on the botting forums for renters. Also was interesting given the claims that -A- leadership is heavily into illicit RMT.

    1. lol oh you mean like the EVESkunk chick did during her presentation? ;-)

  2. I don't think that every alliance bots. I'm sure there are botting players in every alliance, just like there are thieves in every countries (but it don't make the country a thief).

    A "botting alliance" is one that openly encourages botting, helps members to bot, give them tools, info and uses botted ISK as alliance funding.

  3. Oh trust me, we all know a certain lawyer who already got busted RMT'ing moon minerals and system rights. Just as long as your botting doesn't compete with key CCP'ers pet alliances you'll be fine. The same Adahum botters are still around 23 hours a day, 7 days a week two years later. It's too bad their security team doesn't bother with some basic internal security.