URGENT: Recent Forum Crashes and upcoming Old Thread Holocaust

[RCgothic]

Thanks to Mark, we now have a reason for the site crashes we've been having recently:

Our SMF Messages table, essential to the smooth running of the forum, is dangerously over full. The server cannot handle querying all that data, and unless something is done we're going to keep crashing.

We have 2 possible solutions at this time.

#1. Either we can either move to a new, better server (expensive) and appoint a full time database-handler (big job not understatement).
#2. Or we can delete 1/3rd of all posts from the forums and begin a frequent cull in future.

Option one will only be a stop-gap measure if the table keeps growing, and AIFAIK we have neither the money nor a person with enough time or experience on their hands.
That Leaves option 2, which if enacted would mean losing 30,000 posts. As I'm loath to delete old modding threads without a thorough checking, that probably means almost everything in the archive, chat and freepost. Future culls will be taken from suggestions and may come from anywhere on the site.

This thread is a chance to come up with an option three. Failing that, I will now take suggestions on which threads to save.

[Adonis]

I'm OK with option 2.

[limey BSc.]

Possible option #3:

Seperate all topics older than, say, 3 months into another (temporary) forum, volunteers can go through and see whats worth salvaging.
Delete all users who haven't logged in in, say, 1 month.
Impose PM inbox limits like Sovvie did at BCS.

[MLeo]

Possible option #3:

Seperate all topics older than, say, 3 months into another (temporary) forum, volunteers can go through and see whats worth salvaging.
Delete all users who haven't logged in in, say, 1 month.
Impose PM inbox limits like Sovvie did at BCS.

In this case, it's really the message table that has grown really large (so the table that contains each individual post).

[limey BSc.]

Yeah, the second 2 were just additional ways to reduce the load on the server. Obviously, the posts residing in the topics split get split too

[MLeo]

Yeah, the second 2 were just additional ways to reduce the load on the server. Obviously, the posts residing in the topics split get split too

From my understanding, a thread is just a container of posts/messages, if you split a topic, then part of messages get a new container, nothing is duplicated.

[Darkthunder]

From my understanding, a thread is just a container of posts/messages, if you split a topic, then part of messages get a new container, nothing is duplicated.

And splitting 1 thread into 2 does nothing. You still have the same amount of data, just that it's split into 2 pieces instead of 1.

I'll agree (partially) to limey98's suggestion, in that you could begin by deleting inactive users. Furthermore, the "Freepost" section could be made to auto-purge on a monthly basis. Only remove topics who haven't been active for say... 2 weeks. Remove entire threads if no activity has been made within 1 month. No need for all the "in development" modding threads, after the mods have been released. Closed, Archived and after x amount of time, Deleted.

[limey BSc.]

I apologise, I didn't mean split as in the SMF split topic function. More, finding the topics more than 3 months old, fully extracting them and the posts contained from the database and putting them back in in new table that have nothing to do with BCC, but are rather for a temporary forum used for sorting though to find stuff worth keeping.
I just think it would be better than erasing them completely, as there is risk of wiping valuable, good ideas etc.

[ChronowerX_GT]

Option 4, delete the spam archive. And freepost messages that havn't been posted in over a year. That should give you some breathing space.

[PJCham]

How much space is used when posting screen shots directly to the forum? This is the only forum I know of that does it ... usually, pix are linked to a secondary website like Photobucket.

[Jernej]

Seriously guys the reason you posted (posts table too big) is nonsense, i know of a forum with far more posts, topics and members running SMF without any problems whatsoever:
http://forum.sa-mp.com/index.php
 
Perhaps you guys just have a bad hdd in the server or a database size limit.. in any way, big post table is definetly not the cause.
 

[limey BSc.]

Seriously guys the reason you posted (posts table too big) is nonsense, i know of a forum with far more posts, topics and members running SMF without any problems whatsoever:
http://forum.sa-mp.com/index.php
 
Perhaps you guys just have a bad hdd in the server or a database size limit.. in any way, big post table is definetly not the cause.
 

They could have a better server though.

[deanej]

I think stuff that's in the recreational forums category should get priority for deletion.  That category accounts for about 44% of all posts and the forum is about Bridge Commander first and foremost (purging old modding threads that don't have a download or help with modding could also be deleted).

[Darkthunder]

Also, the content of posts make no difference whatseover to the database tables. Images, downloads etc, make absolutely no difference, since it's all just a bunch of text in the database anyways.

As Deanej said, purging stuff in the Recreational Forums first and foremost, is probably the best way to go. If people wanna chat about random nonsense, they can do it over MSN imho, or if this forum gets its own realtime chatbox added instead (with auto-purging after 7-14 days).

[Technerd89]

Seriously guys the reason you posted (posts table too big) is nonsense, i know of a forum with far more posts, topics and members running SMF without any problems whatsoever:
http://forum.sa-mp.com/index.php
 
Perhaps you guys just have a bad hdd in the server or a database size limit.. in any way, big post table is definetly not the cause.
 

your post is so blissfully ignorant... SMF does not simply automatically hold 100,000 posts like magic. obviously the capacity of the server here is at its current limit... how does this translate to a nonsense reason from the site staff? do you think that theyve got something to hide and had to lie about it?

[Rob Archer]

Maybe we should do option 1 anyway..

I mean lets face it there are a few junk threads which are pretty pointless, for some Threads in the modding section for mods that have already been release we can add any Pictures and Files stored in those threads to the End of The Download Thread for the mod thereby Substituting several posts for one extra post, Not to mention threads for projects that are long Dead or locked threads at the end of the day it will make the forum less cluttered which is a good thing regardless.

[Starforce2]

Just don't use the nightsoft server..if you think our problems are bad now, think of what it will be on a server who's max download speeds pokes at 30kps. I don't visit enough to see if it still crashes as often as it used to but it takes forever to download the ships raven sends me.

Plus, I have like 7 pages of PM"s I've been meaning to go through and delete. Drop in the bucket for the grand scheme of things but I shal place it as one of my more important things to do next time I have nothing to do during a commercial.

[Darkthunder]

Might I recommend SiteGround, if getting a different host is the preferred option? I know from experience, they're hosting works very well, and the cost isn't too bad for what is included. Worth having a look at, in my opinion.

[Raven Night]

Just don't use the nightsoft server..if you think our problems are bad now, think of what it will be on a server who's max download speeds pokes at 30kps. I don't visit enough to see if it still crashes as often as it used to but it takes forever to download the ships raven sends me.

Plus, I have like 7 pages of PM"s I've been meaning to go through and delete. Drop in the bucket for the grand scheme of things but I shal place it as one of my more important things to do next time I have nothing to do during a commercial.

Yea, our download speeds wreak and always have. However our up time is around 89 percent now.

The only plus to the Nightsoft server is it is free and has been around for a long time.

Besides, when I retire I wont even have access to it anymore.

[RatPatrol]

Thanks to Mark, we now have a reason for the site crashes we've been having recently:

Our SMF Messages table, essential to the smooth running of the forum, is dangerously over full. The server cannot handle querying all that data, and unless something is done we're going to keep crashing.

We have 2 possible solutions at this time.

#1. Either we can either move to a new, better server (expensive) and appoint a full time database-handler (big job not understatement).

What kind of hardware are we talking here ? .... RAID 5, Mirrored drives .... something with more memory ? ....

Have you have reached the limits for your SQL Install .... I am guessing this site runs on LAMP / WAMP

what are you currently running on ?

Hardware is Cheap, if you know where to look