||Earlier this week, Senior Producer Lance Stites appeared on Reddit to speak with summoners about the upcoming Hecarim patch. The League of Legends community brought lively and informative questions to the table, and we enjoyed the give-and-take they provided. These are all the questions he answered, and we wanted to provide a curated version to all of you.
Sweded: I just saw some post (on reddit somewhere, a pic from PBE) about being able to rightclick a friend from the friendslist to spectate the game they are playing. Care to elaborate if this is some feature that is being implemented in the upcoming patch or something that we weren't supposed to see yet? Thanks!
Lance: Yes, that is a feature that's currently on test on the PBE. However, it isn't expected with Hecarim but I am anxious to see it released with an upcoming patch.
VotedBestDressed: What kind of bugs delayed the patch?
Lance: A few example bugs include:
Significant issues with the surrender function
Some champions occasionally become un-targetable
Champions that undergo any form of transformation will occasionally revert to their previous form, instead of dying. (Example when Kog’maw dies, he transforms into his Icathian surprise, then instead of blowing up and dying, he’s resurrected as normal Kog’maw. A probably more interesting case for this is that players can potentially be transformed into cupcakes permanently via Lulu.)
ApplesFromKira: Will this push back subsequent patches by the same amount?
Lance: Not necessarily, the next patch is not complete. However there's no doubt that we're spending a lot of effort getting this ready, but the release timing isn't directly tied to our Hecarim release.
Melesion: Hello, how did you manage to reduce the downtime of the server(since a couple of patch) ? And what do you think of the 'patch every 2 weeks' format?
Lance: I think the first step was realizing how important it was to our players and sitting down and putting together a team to really tackle the issue.
We started measuring our release times and got our entire company behind decreasing release times.
The team that was responsible for this effort was recognized in front of the entire company.
This does speak to our core value of being player focused.
Raise4Value: What do you do, on a daily base, to ensure the quality of the live-servers?
Lance: Everybody in the company participates in the health of our global service. We have a distribution list where anyone can mail in and report any problems they experience.
We also have a dedicated monitoring team that looks at the technical health of the service. (Capacity, response times, lag)
We monitor your feedback on our forums. Our Player Support team monitors the tickets that you guys submit and alerts us to any major emerging issues.
Any major incident is escalated to the Live Service Team with an escalation matrix that starts with me and if not properly resolved, can be escalated up to the President of the company.
1)How are the developers grouped up at Riot? Do you have a specific group of devs that focus strictly on the core engine/graphics, and another that possibly work on server connectivity and user authentication? Or do all of your developers dabble in everything?
2)How does your build cycle work? Do you do daily internal builds and deployments to QA and Staging environments?
3)How big is your internal QA team? Are most of them gamers, or do they come from a long QA background?
Let's start off with 1.
We've got teams that work on features, champions, skins, and other components of the service. Most of our developers are focused on delivering new features.
For number 2.
This is a high level summary of our overall release process:
A. The development teams creates new features and content (weeks or in some cases months). At some point they hit code-lock.
B. This is where it comes into the release cycle.
C. We begin internal testing on the Release Candidate, and submit bugs for correction.
D. Make sure that all public announcements happen. Conduct a Go / No-Go for PBE release.
E. Review the feedback on PBE and make any further required changes.
F. Take it to live.
Almost everyone at Riot are gamers.
We have a combination of embedded QA that works directly with their teams. There's also a QA team dedicated to the release process.
Our QA testers come from a variety of backgrounds, we have an incredibly high bar for anyone that gets into Riot.
We get over a hundred applications for each of our QA positions and we only select the best. Whether that's the most experienced, or the smartest, there's no such thing as "just a tester" at Riot.
Reilwin: Ooh source control is a good question I'd like to see Lanceed!
Lance: We're currently using Perforce as our primary version control.
IAmYourKing: Can you give us an estimated date/time on when the Hecarim patch will be released?
Lance: We're still working on some big outstanding bugs, so unfortunately it won't be before this weekend.
Drake0727: What type of development do you use when you prepare patches to be deployed into the public/production realm. I assume scrum, but could be wrong.
Also what happens when you recall a patch? Does the users that have the patch installed just rollback to version (new patch ->previous patch), or does a new "patch install" have to be implemented for them to go back a version?
Lance: We use a Riot version of AGILE.
If we revert, it will do a diff between content needed and adjust your client.
Lobos717: Will Varus be included in the next patch with Hecarim? I've been really looking forward to him. (edit, thanks Alexij)
Lance: Varus will come out with the patch following Hecarim.
Dancing_bagel: If you have the time could you sum up the testing process for new patches? Do the QA guys get a specific list of things to try out in a game, or would they go into teams of 5 and play an average game in an attempt to spot any bugs?
Do any spells cause you particular problems, such as Anivia's wall?
Lance: We take it through an internal process, we have daily play tests that the entire company is invited to and participates in. We also have a QA team that follows extensive checklists to test all components and champions, including new content as well as old content.
We've recently started using a Public Beta Environment which allows you guys, the players, to start participating in our releases.
Echono: Out of curiosity, what do you see as being different in the current PBE compared to the old PBE you used to have? Why did you take it down for so long, and then put it back up (relatively) recently?
Lance: The original version was limited in what we deployed. (Didn't have the new champion) We've optimized internal process in order to get content out faster, and this more accurately represents what we actually deploy.
Shadowgripper: How do bugs like "Unable to target champions on the field" or "Unable to surrender" even happen? Are you really changing that much to the core of league of legends?
Lance: Software development, especially ongoing persistent changes can have extensive impact.
Not every change we're making is specifically related to the new content, but can be related to enhancements or new features. (Sometimes even preparing for new features in future releases.)
Nogrid: Are the bugs only centered around Hecarim or do they effect the other balance changes? If they don't effect the normal champion balance is it possible to release that sooner while waiting for the Hecarim bugs to be ironed out?
Lance: No, there are a variety of quality of life bug fixes that still need to happen. The Hecarim champ relies on new technology, so it wasn't an easy task to simply pull him out and deliver him to the players.
Dwhizzle: Lance, first, thanks for the AMA, and keep up the great work at Riot. I have a more over-arching question for you -- When do you personally see the champion releases for LoL beginning to slow down?
I know this has been spoken about before (I believe it was Zileas/Morello) that roughly after 100 champs, champion releases would spread out a little more. Can you shed any insight on this?
Lance: That's more a game design question versus live services.
My job is to release 'em as I get 'em.
Malph1te: is hecraim fun : ) ?
Lance: He's a giant undead centaur...so yeah. :)
Jawtramp: Despite what anyone says: we appreciate it.
Lance: Big thanks. No one's more frustrated than us, and we wanted to take this opportunity to give you guys some insight into the process of getting a champion out. We have been working on these issues non-stop since last week and will continue to do so until it's ready.