TOME Metrics: ~v1.10 RELEASE~

[nehal]

Current Status: v1.10 Released

So, this shouldn't be any surprise to most anyone on IRC, but I'm posting here for better visibility to the general TOME community and to focus discussion and requests around my application - TOME Metrics.


TOME Metrics is a statistics and analysis application I'm working on as a means for developers, both core and addon, to get a better aggregate look at how players play the game.
Player feedback is a fickle thing and very often never given, especially when Addons are considered. How are players building their Sun Paladins? What's the deadliest Character Level? What is killing them? These are questions that can help drive development towards creating a better, more interesting game both by smoothing out rough edges and by clearly flagging game elements that need to be revisited. This application gathers bulk character data and analyzes it to provide a high-level picture of how exactly TOME is played.

TOME Metrics provides the following features:

-A front-end to download character data automatically which matches the standard Character Vault


-A set of simple filtering tools to allow you to easily sift through the data you've downloaded and pin down specific metrics.


-A wide set of analytics:

Game Info
-Version Count
-Campaign(Maj'Eyal, Arena, Infinite Dungeon, etc) Count
-Player Count(by te4.org UID)
-Character Name Count
-Difficulty Count
-Permadeath Count
-Addon Count


Character Info
-Winner Count
-Dead Count
-Size Count
-Antimagic Count
-Min/Avg/Max Level
-Min/Avg/Max Gold
-Sex Count
-Race Count
-Class Count
-Min/Avg/Max for Stats - both Current and Base
-Resource Count


Inscriptions
-Usage of Inscriptions, broken into Infusions/Runes/Other(Taint of Telepathy, etc)


Quest Info
-Counts of Quest status
-Count of Escort Successes


Class Talents
-Talent Tree, by category type, Average Mastery
-Count of many characters invested at least 1 point in a talent
-Median of points invested in a given talent
-Mode of points invested in a given talent


Generic Talents
-Talent Tree, by category type, Average Mastery
-Count of many characters invested at least 1 point in a talent
-Median of points invested in a given talent
-Mode of points invested in a given talent


Prodigy Stats
-Count of Prodigy usage broken out by statistic/tree


Death Info
-Deadliest NPCs
-Deadliest Character Levels


Class-Race Stats
-A grid to analyze class-race combination counts


Offense Stats
-Min/Average/Max for Mainhand, Offhand, Spell, Mind stats
-Min/Average/Max for Damage Bonuses(+10% Fire Damage) and Damage Penetration


Defense Stats
-Min/Average/Max for Defensive Stats(Armor, Defense, Etc)
-Min/Average/Max for Resistances
-Min/Average/Max for Immunities


Achievements
-Achievements Gained


Misc Stats
-Min/Average/Max Speeds
-Min/Average/Max Vision Stats
-Count of Telepathy Types
-Min/Average/Max Healing Stats

Download Current Version
You can download the latest version at http://efbnet.com/tome/TOMEMetrics-1.10.7z
See THIS POST for release and version details.


Thanks
*A huge thanks to DarkGod for making some changes that enabled me to do this - without his support for this project it never would have happened!

Questions, comments, and suggestions are all very welcome and can be posted here or emailed to tome@efbnet.com

[Marson]

Awesome, nehal! A nice boon to stat junkies. Downloading...wait... no download! Must have!

[edge2054]

Really anxious for this Should be a great help in figuring out which talents need the most attention.

[Doctornull]

This is such a neat looking tool. Thanks!

[nehal]

Case Study: Draconians

Summary: AuraoftheDawn grabbed me in IRC and wanted to know a bit more about how players are playing his Draconian race. Specifically, how they're building their racial talent tree. Draconians are an interesting sample set because of the history of the race. The original maintainer built them(addon shortname 'draconic') and then when Aura took over he gave his work a new shortname(draconic-fork), and then there is another unknown fork in the vault as well(draconicunlock). Draconians also provide an interesting glimpse at the addon-neutrality of TOME Metrics with the ability to automatically handle non-standard Addon behavior - in this case the nature of Draconians having a *very* divergent racial tree, something not seen in standard TOME.


Step One: Getting the characters


I'm doing this from a fresh install of 1.0, with just a few minor changes over yesterday's build. I launch TOME Metrics and click "Get More Characters" to open the Fetch Window.
I set my filtering to look ONLY for characters who are of the Draconian race and have the draconic-fix addon active.
When the application finishes fetching all matching characters from the vault, it confirms with a total and I am good to Close the window.
For reference these ~2000 characters took around maybe an hour to download and take up around 100MB of disk space.

Step Two: Loading the Characters


The current application behavior is that when characters are fetched, they're stored to disk and *not* loaded immediately, so once we're back at the main screen we simply need to click "Load from Disk" and the application reads all of our stored data. This was all done on a fresh install and so there's nothing besides the ~2000 draconians we just fetched. Once this is done, we'll see our Filters & Totals populate and know we're good to have the application analyze the loaded data.
*Please note there is a difference between the Fetch Count and the Totals Count - I'm still working to validate(hard when dealing with thousands of files!) but I believe most of these discrepancies are due to a bug in the character vault that occurs with certain localized characters.


Step Three: Apply filters


The filters that load are based on all characters currently loaded, so we're able to see here that all the loaded characters are Draconian(no other options) but that we have a wide spread of Classes, Campaigns, Difficulties, and other addons used by these characters! I want to see a total group analysis, so I click 'Apply Filters' without making any selections and when the application is done with it's analysis, it populates all the fields in the Filtered Results pane. We can then quickly see a breakdown of many of the standard Filters and how many times they were used. To answer Aura's question, I move over to the Talents(Generic) tab.

Step Four: Generic Talents Pane


On this view, I see a breakdown of all Generic Talents, grouped by Type(Technique, Race, etc) and then Tree. Each individual talent gives us the following information:
Average Mastery: the Mean talent category mastery level across all filtered characters.
Times Taken: How many of the filtered characters took a given talent. This means having *a nonzero* number of points invested. 1 point or 5 points, both would get tallied.
Median Level: Provides the Median level of the talent for characters that took it. I considered using Mean here, but realized that a mean level of "2.48" doesn't necessarily say much and thus went with Median instead. High school math reminder: Median is the MIDDLE sample of a given set of data: (1,1,2,3,5) would have a median of 2, despite it's average being 2.4.
Mode Level: Provides the Mode level of the talent. The most common number of points invested - in the set (1,1,2,3,5) the mode would be 1 - indicating that the MAJORITY of these characters decided this was the optimal level.


I leave the actual analysis of what all this means to Aura, I'm not familiar with the addon in the least. There seems to be a discrepancy in that the at-birth talent Channel Power is missing from ~100 of these characters. In a case like that I as an end-user would start looking at some of the additional info to see if I could find matching numbers(are the number of characters missing that talent equal to the number using a specific addon?) and/or applying some additional filters to try and narrow down the sample set that's not matching up. Worst case I would post here and hope that nehal can replicate my results and do a data-audit to see if there's something else going on!


I hope this has been informative and I will post more of these as additional developers let me know what they're interested in seeing!

[nehal]

Case Study: Chronomancers

Summary: Edge is working on a huge and sounds-like-awesome Chronomancer rework and wanted to get an idea of what players were *currently* doing with Chronomancer talents. So off we go!


1. Grabbing Chronomancers


We worked out what the valuable information would be:
All Paradox Mages and Temporal Wardens
From versions 1.1.5 to 1.2.2
Levels 25-50
Playing the Maj'eyal Campaign

This required a bit of clicking to change up the filters for class/version between pulls, but the sample set is surprisingly small(holy crap these classes need some love!) so it was quick to grab the data.

2. Some overview info


We can see that there's a huge shortage of winners in these classes. We can also see that the 20s being a hugely dangerous level range for them.

3. Talents


chrono-results.xlsx
Using the filters for actual talent analysis, I look at TW and PM separately, and was able to easily copy-paste the talent grids into Excel

[darkgod]

I applaud!

[nehal]

Case Study: Afflicted

Summary: I like the Afflicted Classes, thematically and gameplay style I think they're fun classes that provide a very neat power curve - being reasonably fragile at the start and then slowly becoming increasingly terrifying and unstoppable. But they aren't popular classes, and they have a lot of very rough edges, so I wanted to take a look and see what was right/wrong with them. To do so I have pulled all 1.1.5 Winners - 1.1.5 because it was a long-running stable version and there haven't been any serious balance changes since then, so it's still a comparable set of data; Winners, because I'm curious what makes these classes successful - the corpses of a thousand dead level 1 characters might say something about early game difficulty, but it doesn't say much about what builds are successful. And winners, in aggregate, will tend to gravitate towards optimal builds.


Curious Findings
Cursed - 70 Winners

• Cornac was the overwhelming choice for race, with 48(~69%) taking it. Stats can't say why, but I doubt there's any disagreement that it's for Level 1 Rampage unlocking
• Maybe surprisingly, 53(~76%) went Antimagic. This is often seen as a controversial choice, since Cursed are so Generic Point starved.
• Superpower was the most popular prodigy, being taken 53(~76%) times. I Can Carry the World! came in a distant second at 23(~33%) buy-ins.
• Cursed are super powerful endgame and this appears to lead to a bit of hubris on the part of the players: The deadliest level for these winners was 50, with 150 deaths. Second place goes to level 32, with 135 deaths.
• This is likely because the top killer was an overpowered greater multi-hued wyrm, with 180 total playerkills. After that is Armoured Skeleton Warrior but only with 68 playerkills.
• Only 26(~37%) of these characters took the Cursed Aura tree! One character by accident possibly, since they didn't invest in Dark Gifts. Except for that, those that did take the tree 5/5'd Defiling Touch and Dark gifts. But only 6 players took Ruined Earth and 5 took Cursed Sentry. The latter will be interesting to watch in 1.2.x, since it no longer destroys the weapon! The divide here is between a 5/5/0/0 build and a 5/5/5/5 build.
• The Cursed Form tree obviously needs some work. Only 46(~66%) of players invested all the way to Grim Resolve. And the clear preferred build was 1/5/1/1.
• All skills in Combat Training, when they were taken, went to 5/5. Surprising no-one, all characters took Thick Skin and Armor Training. Only 65 took Accuracy, 59 took Weapons Mastery, and 6 took Dagger Mastery. The differences here are likely due to the fact that Cursed make great mindstar/Psiblades characters, which frees up Generic points that would normally go into Accuracy and Mastery. Even without Psiblades, they work well with both 2H and dual-wielding builds, so the daggers aren't terribly surprising.
• Speaking of Psiblades, 17 characters took the tree but numbers dwindle for continuing skills down to only 9 users taking Nature's Equilibrium and even then, 5/1/1/1 was the preferred build.
• The 53 Antimagic characters saw 49 of them take Antimagic Shield but only 22 took Mana Clash. 1/1/5/1 was the preferred build.
• Only 48 of those characters took Fungus, 42 of them took all four skills, and 5/1/5/1 was preferred. Cursed naturally buff Mindpower, so that turns into free almost non-stop regens in the lategame.
• The most consistent Class Tree was Gloomed, with all 70 Characters investing in all 4 skills. 5/5/5/5 was the preferred build.
• On the opposite end of the spectrum, Fears was only taken by 9 characters, with only 5 actually picking up all four skills. 5/5/5/5 was preferred, though, which makes sense with how the skills synergize. It is, however, a pointed fact that this tree could really do to be substituted for something new and a little more Cursed in nature.
• 29 characters took Predator, 23 of them all the way to Mimic. 5/1/5/5 is the popular build here. I'm actually surprised by the high numbers here, since I don't personally like the tree, but especially in the East you can just choose an Orc and have a guaranteed damage buff against the vast majority of the enemies in the lategame.
• 68 Characters took the first three skills in Rampage, but only 22 took Slam. Even then, 5/5/5/1 was the build of choice. Slam sucks. Nobody is surprised.
• Of the three mutually exclusive Capstones, we see Surge being taken 50 times, Cleave taken 24 times, and Repel taken 8 times. When these skills are taken, they are 5/5'd reliably.

Doomed - 26 Winners
Wow the sample size is even smaller. Looking at 1.1.5 Winners as a whole, Doomed are the *least successful* of the standard(non-addon) classes!

• Cornac are the popular race here, but not nearly by the same margin - only 9 characters. 6 characters were Halflings!
• 19 Doomed went Antimagic. Much like Cursed, this seems high to me since Doomed are Generic hungry as well - but Antimagic Shield and Deflection do work well together!
• Doomed inspires a different kind of overconfidence in its players: Level 50 is still the most lethal(69 Deaths) with Level 20 far behind(23 Deaths
• The top killers of those endgame Doomed are Elandar(18 Kills) and Atamathon(!!!)(17 Kills)
• Mental Tyranny was taken 22 times, the most popular Prodigy. Spell Feedback and Eye of the Tiger are tied for second, being taken 8 times each.
• 25 of these Doomed took Thick Skin - to 5/5 of course - and 23 even took Armor Training, but only a point. Weapons Masteries were ignored completely.
• We see the same sharp cliff edge in Antimagic - 18 went as far as Antimagic Shield but only 9 took Mana Clash - and 1/1/5/1 was the popular point distribution.
• 15 Characters took Fungus as well, 11 all the way to Sudden Growth. 5/1/3/1 was the buy-in of choice.
• 15 also took Psiblades and 5/5 was preferred there, but only 6 took Thorn Grab.
• 7 of these Doomed took Cursed Aura, and much like Cursed those that did took both Defiling Touch and Dark Gifts to 5/5. Ruined Earth and Cursed Sentry were ignored much more, though, being taken twice and once, respectively, and only a point in all cases.
• Just like with Cursed, 1/5/1/1 in Cursed Form was preferred.
• The Feed Tree needs a lot of love. Everyone took Feed, but only 17 completed the tree and 1/1/1/1 across the board was preferred. The scaling on the skills is pretty terrible all around, so this isn't a huge surprise.
• Gestures are also mostly ignored. 22 characters took it so far as the third skill but only 14 took the fourth. 1/1/5/1 was the build of choice. This makes sense since the third skill is free mindpower. The fourth skill being taken at all is surprising, it's got terrible scaling and doesn't mitigate any real amount of damage.
• Fears *REALLY* need some love. Only 8 Doomed took the tree, and only 2 to the end. 1/5/1/1 was the preference here. Given that neither of the two classes that has the three want to use it, something is clearly very wrong here.
• Punishments was the constant for all characters. Only a single Doomed ignored the final skill, and 5/5/5/5 was the popular choice.
• Force of Will is very similar. Everyone took Willful Strike and Deflection, and 22 people went so far as to get Unseen Force - 5/5/5/5 all around.
• 24 players used Darkness, though only 7 took Dark Tendrils. 5/5/3/5
• 25 players bought into Shadows with a slight dropoff as only 18 took Focus Shadows. 5/5/5/1 is the stock-standard build for Shadows and this shouldn't be a huge shock.
• Only 10 players took One With Shadows, and only 6 to the point of Shadow Decoy. 1/1/3/1 was the build of choice. Giving Doomed Shadows more power by adding an advanced tree was a good start, but I don't think the benefits are quiet attractive enough to players to really invest. The three points in Transposition is because that's the soft cap for the number of debuffs that get removed.

[HousePet]

None of those Afflicted details surprise me at all.

[nehal]

Case Study: Wyrmics

Summary: Astreoth asked in IRC for me to do the same thing for Wyrmics that I did for Chronomancers up above!

Sample Set: I pulled all Wyrmics between the levels of 25-50 from 1.1.5 to Current, for the Maj'Eyal Campaign. This comes to a total of 419 characters.

General Info:

• 19 Yeek Wyrmics? What the hell.
• A lot of deaths in the midgame and midgame bosses(look at all those midgame bosses in the top 10!)
• This coincides with our average character level being 34 and only around 10% of these characters being winners.

Class Talent Distributions


wyrmics-talents.xlsx

[astreoth]

thanks dude

[Doctornull]

Any chance you could run analytics on an addon class? I'd like to see how the Creep compares to its Doomed and Cursed compatriots.

[nehal]

Case Study: Creep

Summary: Oh just look at the previous post. I pulled *all* Creep characters that played maj'eyal. No level or version restrictions.


Preliminary Findings
-The full charset is still pulling, but this class is either very non-viable at high levels or unattractive to experienced players. Less than 2% of all Creeps played are character level >=25!
-While 310 players have played at least 1 Creep...
-User 114339 accounts for just over 10% of all Creeps played
-And 4 more players make up another 10%
-There are no Creep winners
-The average level of Creeps is 2.59 - suggesting that they are played for a couple of levels but then dropped
-The top three races are: Cornac(139), Halfling(125), Faerie(84)

Deaths

-New as of last night's build, I'm also now tracking 'final' deaths(the last death record on a character) - this should be read from the data as either the character's final death OR the point at which a player has abandoned a character.
-There are 716 total deaths tracked for these characters: 301 at level 1, 175 at level 2, and then a drop to 35 at level 12.
-The Final Deaths mirror this - with most players ending their characters at Level 1, followed by Level 2.

Prodigies

-Nothing too much to see here, the same standard Prodigies are still used regularly

Talents
-I feel wholly unsuitable to analyze a class I didn't realize existed until an hour ago. Here's a spreadsheet containing all relevant talent info!
creep-talents.ods

[Doctornull]

-There are no Creep winners

Creepers never win

Talents
-I feel wholly unsuitable to analyze a class I didn't realize existed until an hour ago. Here's a spreadsheet containing all relevant talent info!
creep-talents.ods

I'll take a look, thank you!

[Razakai]

I'd be interested to see analytics on Necromancers using my addon, so I can figure out what talents are in need of improvement.

http://te4.org/games/addons/tome/eternaldarkness