29 09 2007
Old scores are now purged from the Frets On Fire Charts
Lately our web host has been somewhat concerned about the database load generated by the Frets On Fire World Charts web application. While the app does employ a number of caching schemes, it executes some pretty large queries in the process, which are reportedly hindering other users of the system. Given the choice of shutting down the service or mitigating the load, we opted for the latter by adding a process that removes old scores from the database.
The charts now list the best 50 scores for each song and expire all others that are at least three weeks old. The expired scores will disappear from all views, including the personal player pages. This is a pretty aggressive scheme, but it is needed to keep the amount of data bounded. Note that instead of deleting the scores, we are simply moving them to an offline table.
Next we’ll monitor how well this scheme performs in practice and tune it accordingly. Please post your questions and suggestions in the comments below.
13 Comments so far
Leave a comment
Hey guys,
I don’t like the new system. It’s not really losing 70 million total score (but that still hurt!). My local high scores are pretty crowded (some songs have 50 point difference between my best and fifth score), so it is difficult for me to get on my local list (hence submitting to the world charts). I don’t like the idea of removing my local scores, I rather not care about the World Charts.
It’s not really clear how the new system works like, the description says “charts now list the best 50 scores” and “expire all _others_ that are at least three weeks old”. I’m pretty sure my 40 or so disappeared gold medals were in the best 50. I also still have scores dated back to May in my personal list.
BTW, any plans on dealing with cheaters?
I will think of something more constructive instead of whining, so stay tuned for another mail ;]
Inorix
Hi Inorix,
I understand that you’re upset, and rightly so. The reason for these rather drastic measures was that we needed to do something now to avoid having to shut down the charts completely. We hope to somehow account for the expired scores in the future, which is why they were saved to a backup table instead of simply being deleted. At this point, thought, it isn’t yet clear how that will be done, and suggestions are welcome.
Unfortunately these days we haven’t got much time to work on the charts server, so proper anti-cheating measures and other improvements might not happen very soon. After all, we need to allocate some time for our next game too
Hi,
just an advice: I understand the reasons for this choice, but you should at least notice the users with a sort of “news” tab in the charts pages. When old scores were deleted I was quite p**sed off, not because of the lost data but mainly because no reason was given… we users didn’t know if it was due to a bug, if the data were going to be recovered or not, in the end we didn’t know anything, we just saw the scores disappearing. That was really annoying. Now I can find the answer in your blog (which I see today for the first time :)) and can understand your motivations, but I suggest you to give some feedback in the official charts web pages too.
PS: oh, another point: why did you remove the “progressive” number in the personal scores page? It was useful to check if you were near to the top with a song or not. Actually I liked a lot that feature, even if you probably consider it a bug since the progressive number was related to the song scores and not to the player ones
Hi Piccia,
Thanks for your comments. I’ve now added a link to this blog entry from the charts site. The global score position, or the progressive score number as you called it, was actually one of the worst database chewers in the system, so it was first to go. I agree that it would be a nice feature, but some pretty heavy optimization needs to be done before it returns.
Why not just show the 50 best and delete smaller scores.
aka if someone is on 50th and someone and someone gets one more point. knocks him of the list and deletes his record.
Just my opinion.
Daniel,
That’s pretty much how the current system works, with the added restriction that the removed records must be a few weeks old.
Hello!
I understand the reasons why you had to move all the “spare” records, and I am agree with you. You can’t just let the DB grow and grow and grow. I have “lost” scores, as everybody, but think about it: soon or later, it had to happen. It’s nice to purge some old records nobody is fighting for, and this encourages people to fight for the main 50 positions.
Big up to you all!
See you fretsonfiring.
I agree that the database needs trimming, but I’m still a little confused as to why my scores have been removed. Before, I had several medals, now I have 0 points and no songs played on my scores page. I decided to check up!
Quick Example; Misirlou, my score in supaeasy = 70502. I think that should be third place, above the guy with 69747… definitely top 50!!!
I just lost a gold medal as well … at least all the cheaters are still there. Sigh.
i have to admit, that i think i don’t understand the removal concept properly. aren’t you saying, that the charts: “list the best 50 scores for each song and expire all others that are at least three weeks old”. the reads to me, like the best 50 songs are kept and the rest is removed!?! but why than are also people removed, who are leading the charts of a certain track. wouldn’t it be a nicer removal scheme, where at least the data of the top 10 players of a track is never removed?
greetings
I am fine with the removal of old records from the database. It is sensible, and reasonable… but, if some of the reports above are true… it’s buggy.
I like that, when I search for my player name, I can see which songs I have medals on. In addition, I would very much like to see which songs I’m fourth on… those ones where, with just a little extra push, I might get a medal :). In fact, if the rank is above 50, it’d be cool to see it (ranks below 50 being less meaningful in the current system).
Hi people!Maybe this is not the right place to send comments about this I wanna ask,but i have no choise.I’ve downloaded Frets on Fire, and I want to import songs What I have to do???Please someone give me advice because I want to play this game it’s fantastic. Later
check: http://keyboardsonfire.net/