Bridge Commander Central
Recreational Forums => Forum Discussion => Topic started by: RCgothic on February 03, 2009, 04:54:45 PM
-
Thanks to Mark, we now have a reason for the site crashes we've been having recently:
Our SMF Messages table, essential to the smooth running of the forum, is dangerously over full. The server cannot handle querying all that data, and unless something is done we're going to keep crashing.
We have 2 possible solutions at this time.
#1. Either we can either move to a new, better server (expensive) and appoint a full time database-handler (big job not understatement).
#2. Or we can delete 1/3rd of all posts from the forums and begin a frequent cull in future.
Option one will only be a stop-gap measure if the table keeps growing, and AIFAIK we have neither the money nor a person with enough time or experience on their hands.
That Leaves option 2, which if enacted would mean losing 30,000 posts. As I'm loath to delete old modding threads without a thorough checking, that probably means almost everything in the archive, chat and freepost. Future culls will be taken from suggestions and may come from anywhere on the site.
This thread is a chance to come up with an option three. Failing that, I will now take suggestions on which threads to save.
-
I'm OK with option 2.
-
Possible option #3:
Seperate all topics older than, say, 3 months into another (temporary) forum, volunteers can go through and see whats worth salvaging.
Delete all users who haven't logged in in, say, 1 month.
Impose PM inbox limits like Sovvie did at BCS.
-
Possible option #3:
Seperate all topics older than, say, 3 months into another (temporary) forum, volunteers can go through and see whats worth salvaging.
Delete all users who haven't logged in in, say, 1 month.
Impose PM inbox limits like Sovvie did at BCS.
In this case, it's really the message table that has grown really large (so the table that contains each individual post).
-
Yeah, the second 2 were just additional ways to reduce the load on the server. Obviously, the posts residing in the topics split get split too :P
-
Yeah, the second 2 were just additional ways to reduce the load on the server. Obviously, the posts residing in the topics split get split too :P
From my understanding, a thread is just a container of posts/messages, if you split a topic, then part of messages get a new container, nothing is duplicated.
-
From my understanding, a thread is just a container of posts/messages, if you split a topic, then part of messages get a new container, nothing is duplicated.
And splitting 1 thread into 2 does nothing. You still have the same amount of data, just that it's split into 2 pieces instead of 1.
I'll agree (partially) to limey98's suggestion, in that you could begin by deleting inactive users. Furthermore, the "Freepost" section could be made to auto-purge on a monthly basis. Only remove topics who haven't been active for say... 2 weeks. Remove entire threads if no activity has been made within 1 month. No need for all the "in development" modding threads, after the mods have been released. Closed, Archived and after x amount of time, Deleted.
-
I apologise, I didn't mean split as in the SMF split topic function. More, finding the topics more than 3 months old, fully extracting them and the posts contained from the database and putting them back in in new table that have nothing to do with BCC, but are rather for a temporary forum used for sorting though to find stuff worth keeping.
I just think it would be better than erasing them completely, as there is risk of wiping valuable, good ideas etc.
-
Option 4, delete the spam archive. And freepost messages that havn't been posted in over a year. That should give you some breathing space.
-
How much space is used when posting screen shots directly to the forum? This is the only forum I know of that does it ... usually, pix are linked to a secondary website like Photobucket.
-
Seriously guys the reason you posted (posts table too big) is nonsense, i know of a forum with far more posts, topics and members running SMF without any problems whatsoever:
http://forum.sa-mp.com/index.php
Perhaps you guys just have a bad hdd in the server or a database size limit.. in any way, big post table is definetly not the cause.
-
Seriously guys the reason you posted (posts table too big) is nonsense, i know of a forum with far more posts, topics and members running SMF without any problems whatsoever:
http://forum.sa-mp.com/index.php
Perhaps you guys just have a bad hdd in the server or a database size limit.. in any way, big post table is definetly not the cause.
They could have a better server though.
-
I think stuff that's in the recreational forums category should get priority for deletion. That category accounts for about 44% of all posts and the forum is about Bridge Commander first and foremost (purging old modding threads that don't have a download or help with modding could also be deleted).
-
Also, the content of posts make no difference whatseover to the database tables. Images, downloads etc, make absolutely no difference, since it's all just a bunch of text in the database anyways.
As Deanej said, purging stuff in the Recreational Forums first and foremost, is probably the best way to go. If people wanna chat about random nonsense, they can do it over MSN imho, or if this forum gets its own realtime chatbox added instead (with auto-purging after 7-14 days).
-
Seriously guys the reason you posted (posts table too big) is nonsense, i know of a forum with far more posts, topics and members running SMF without any problems whatsoever:
http://forum.sa-mp.com/index.php
Perhaps you guys just have a bad hdd in the server or a database size limit.. in any way, big post table is definetly not the cause.
your post is so blissfully ignorant... SMF does not simply automatically hold 100,000 posts like magic. obviously the capacity of the server here is at its current limit... how does this translate to a nonsense reason from the site staff? do you think that theyve got something to hide and had to lie about it?
-
Maybe we should do option 1 anyway..
I mean lets face it there are a few junk threads which are pretty pointless, for some Threads in the modding section for mods that have already been release we can add any Pictures and Files stored in those threads to the End of The Download Thread for the mod thereby Substituting several posts for one extra post, Not to mention threads for projects that are long Dead or locked threads at the end of the day it will make the forum less cluttered which is a good thing regardless.
-
Just don't use the nightsoft server..if you think our problems are bad now, think of what it will be on a server who's max download speeds pokes at 30kps. I don't visit enough to see if it still crashes as often as it used to but it takes forever to download the ships raven sends me.
Plus, I have like 7 pages of PM"s I've been meaning to go through and delete. Drop in the bucket for the grand scheme of things but I shal place it as one of my more important things to do next time I have nothing to do during a commercial.
-
Might I recommend SiteGround (http://www.siteground.com), if getting a different host is the preferred option? I know from experience, they're hosting works very well, and the cost isn't too bad for what is included. Worth having a look at, in my opinion.
-
Just don't use the nightsoft server..if you think our problems are bad now, think of what it will be on a server who's max download speeds pokes at 30kps. I don't visit enough to see if it still crashes as often as it used to but it takes forever to download the ships raven sends me.
Plus, I have like 7 pages of PM"s I've been meaning to go through and delete. Drop in the bucket for the grand scheme of things but I shal place it as one of my more important things to do next time I have nothing to do during a commercial.
Yea, our download speeds wreak and always have. However our up time is around 89 percent now.
The only plus to the Nightsoft server is it is free and has been around for a long time.
Besides, when I retire I wont even have access to it anymore.
-
Thanks to Mark, we now have a reason for the site crashes we've been having recently:
Our SMF Messages table, essential to the smooth running of the forum, is dangerously over full. The server cannot handle querying all that data, and unless something is done we're going to keep crashing.
We have 2 possible solutions at this time.
#1. Either we can either move to a new, better server (expensive) and appoint a full time database-handler (big job not understatement).
What kind of hardware are we talking here ? .... RAID 5, Mirrored drives .... something with more memory ? ....
Have you have reached the limits for your SQL Install .... I am guessing this site runs on LAMP / WAMP
what are you currently running on ?
Hardware is Cheap, if you know where to look
-
So what's the problem with purging recreation forums?
Freepost could be completely purged (You guys would fill it up again in a month)
And the others could have discussion not made in 2-3 months removed.
There's no need to upgrade servers or spend more money. It's like cleaning out your closet. Sometimes you have to get rid of the old sentimental crap that serves little use.
EDIT: I just look over the forum list and notice that Freepost has over 20 thousand posts, making up nearly a third of all discussion on these boards. Most of these threads last little more than a week before a new one is created.
-
Sum1 say culling?! Oh noez, the wraith are comming!!
[serious business]As much as I would would rather not see old threads go, I don't have the finances to support BCC in it's endeavours to find a new host. :( I move for a deletion period of 3 month inactivity for spam threads, 6 for modding, and no auto deletetion for scripting/tech support/forum discussion.[/serious business]
Radio Sheppard, we gotz sum wraith 2 kill!
-
I've seen several forums with a page cap rule.
After 50 or so pages the thread got locked and a new one was opened.
So there would be "Somewhat useful facts thread 2" "Somewhat useful facts thread 3" etc...
That has solved the problems for them.
-
I stick with Kirk on the Deletion Period, those sound really better for me than a 2/ 3 month inactiity- Killautomatik. (Heck 2 months can go so fast!)
A PM Box limit like limey suggested may be also get some freespace on the server.
-
I too am for the pm limitation. I know that quite a few of mine contain uses things.
-
I know im just a lurker, but one of the forums I used to go on had this problem, as well as purging all the old posts, they also cleansed anyone who hadnt posted in atleast a couple of months.
-
Hence the lurker now posts. :P
-
I still think we can handle this by deleting some old crap in freepost. Start the stickied topics over, and delete anything past the first page. Hell, the first page dates back to July of last year, anyways. Is anything in that forum truly archive worthy compared to shelling out more money and time on a new server?
-
It may be better to do voluntary housekeeping 1st.
If everyone can go through their PM's and delete anything they no longer need.
By the way folks, please watch the tone of your posts, there has already been one reported.
-
Yes, voluntary pm deletion would be a start.
-
Already done.
-
By the way folks, please watch the tone of your posts, there has already been one reported.
My current curiosity is piqued. :D
PM's are going. Except for the important video making ones :D.
-
What about putting the info in the solved problems section into a downloadable file and posting it on BCFiles. The solved problems section could be deleted here and have the info retained on BCFiles. Also getting rid of old topics that haven't had a post in about 3 months is something I agree with, along with the free post.
I'd also recommend getting rid of all of the old Mod of the Month threads and deleting them on a regular basis as time goes by.
-
If you delete the solved problems, what's to keep the noobs from posting them over and over? We lose our typical "Use Search Button" response. :P
-
Well, I for one am down with voluntary PM deletion. I just deleted 28 pages of PM's dating back to late 2006. Hooray for this post that now fits!
-
Ahh, PM deletions... that's something I've been meaning to do. *runs off to do so*
Edit: And 9 pages of PMs are gone *happy dance*
-
I deleted 6 pages of PM's, but kept only the ones for the last month.
-
If you delete the solved problems, what's to keep the noobs from posting them over and over? We lose our typical "Use Search Button" response. :P
Create a locked, stickyed thread that says LOOK HERE FOR PROBLEMS and tell them they're on BCFiles. I don't know it was just an idea.
-
i got rid of all my PMs as they were from last year and even the year before that 3 pages worth hope that helps
-
Ok, I'm going to start deleting threads tomorrow.
Most of Free-Post, Chat and the Archive will go. If there's anything anyone specifically wants saving, now's the time to mention it.
-
We have a forum for spam, please use it. El.
-
We have a forum for spam, please use it. El.
-
If you delete the solved problems, what's to keep the noobs from posting them over and over? We lose our typical "Use Search Button" response. :P
Create a locked, stickyed thread that says LOOK HERE FOR PROBLEMS and tell them they're on BCFiles. I don't know it was just an idea.
I think we can pick which forums do we have a purge and which not to.
-
just have to keep the sexiest women thread
-
just have to keep the sexiest women thread
Dude, its a sticky...its not going anywhere.;)
-
just have to keep the sexiest women thread
Dude, its a sticky...its not going anywhere.;)
Lol, too right!
-
mug shots thread would be nice to keep to.
-
mug shots thread would be nice to keep to.
Ditto!
-
id like to keep useful facts thread, maybe desktop pics thread?
the stickied "Welcome to Freepost" thread could go, tho, im thinking...
-
Along with the rest of the forum.
*hides*
-
Thanks for the offer Raven, but I don't think that will be necessary.
The archive has just been nuked. I'm about to do chat and freepost. Threads not posted in within 30 days will go, except for stickies. I'll sticky the threads mentioned for the moment so that they're safe.
-
Ok, we're down to 65k posts, which should be enough for the moment. I will still be looking for threads to trim.
I'll make a thread taking suggestions for deletions, particularly from the modding forums where others will know better than I what to delete or not.
I suspect the mugshot thread is dead, which will have been an unfortunate accident. I thought it was in chat, but couldn't find it after the culling of free-post. That and any other threads, such as Post Spam Here or BCC Bump thread can be restarted.
-
Ok, we're down to 65k posts, which should be enough for the moment. I will still be looking for threads to trim.
I'll make a thread taking suggestions for deletions, particularly from the modding forums where others will know better than I what to delete or not.
I suspect the mugshot thread is dead, which will have been an unfortunate accident. I thought it was in chat, but couldn't find it after the culling of free-post. That and any other threads, such as Post Spam Here or BCC Bump thread can be restarted.
For BC Modding, don't bother keeping my D'Kyr or Serenity threads. The only one of major importance is the Mithra, for me. I can always start up the other two at another date when I actually put some work into them for a change. I'd like to mention that in my humble opinion, the tutorials area and the topics within it should be saved, though I'd also like to suggest trimming any useless posts out of them to keep them clean and efficient. jsut my two cents for today. :)
-
Im not sure pruning would help, as the forum has gone down a few times already. We had the same kind of problem at BCU...abiet caused by hacking or bad mods installed.
Here is some info that may help.
You can do this one of two ways.
Login to phpmyadmin.
Go into your database.
Check the smf_messages table, and select the option to repair it.
OR you can login to phpMyAdmin and run the following queries:
REPAIR TABLE `smf_messages`
OPTIMIZE TABLE `smf_messages`
You should run the smf_messages table repair at least once a week.
-
Yeah as far as I know, pruning has been tried, and still getting the same errors occasionally. Might be a repair needed.
-
i hate to say but the most recent crash might have been hacking casue a few members of the IKF clan experienced crashes on their own PCs due to hacking around the same time BCC went down yesterday
-
it definitely wasnt caused by hacking...
-
there goes that theory lol
-
there goes that theory lol
you try. *ruffles your hair*
-
Well, it is possible to overflow the table, but it would be an odd way to hack the forum. Most attackers overflow the connections table instead...so it is unlikely to be a hack. Probably a damaged table.
-
Here is some info that may help.
You can do this one of two ways.
Login to phpmyadmin.
Go into your database.
Check the smf_messages table, and select the option to repair it.
OR you can login to phpMyAdmin and run the following queries:
REPAIR TABLE `smf_messages`
OPTIMIZE TABLE `smf_messages`
You should run the smf_messages table repair at least once a week.
Yeah as far as I know, pruning has been tried, and still getting the same errors occasionally. Might be a repair needed.
Thats very basic stuff guys, those options were exhausted some time ago now.
-
Then perhaps its a duplicate key hiding in there somewhere in the table. Did you do an error search?