|
Welcome to the International Skeptics Forum, where we discuss skepticism, critical thinking, the paranormal and science in a friendly but lively way. You are currently viewing the forum as a guest, which means you are missing out on discussing matters that are of interest to you. Please consider registering so you can gain full use of the forum features and interact with other Members. Registration is simple, fast and free! Click here to register today. |
3rd August 2014, 12:30 PM | #161 |
Join Date: Sep 2007
Posts: 10,924
|
You may know something I do not, but the forum seems to work great when it is not having lag issues. So if those "severe performance issues" really aren't just improper hardware or software setup, then I would agree with you. As for mobile users, that is what we have tapatalk for. As for security updates, are you sure those are even being added to this forum now, let alone when it stops in three years? If you do take on the task eventually I wish you the best of luck. Also, thanks for all of the server statistics in post #144!! |
__________________
________________________ |
|
3rd August 2014, 12:47 PM | #162 |
Technical Admin
Join Date: Jul 2006
Location: Canada's Texas
Posts: 1,495
|
In addition to this, there is a custom script for importing big boards onto Xenforo. The built in script works (I've run it a few times as tests), but it by no means fast.
There is a custom import script that might be worth considering: https://marketplace.digitalpoint.com...in-4.1535/item Or there are even people who specialize in such things: http://xenforo.com/community/threads...ervices.55328/ If donations are part of the equation allocating some to something like that might be worth doing, when that time comes anyway. |
__________________
One man's reason that something is not reliable evidence is another man's whine about how others won't buy 3 magic beans with the family cow. - hgc |
|
3rd August 2014, 12:58 PM | #163 |
Technical Admin
Join Date: Jul 2006
Location: Canada's Texas
Posts: 1,495
|
Huh, that's not nearly as bad as I'd thought, it's not a VPS anyway. RAM isn't too bad, though how it's allocated might be an issue (i.e. if MySQL is only getting 2GB). How much is given to Apache might be part of it as well. CPU could be part of it, and disk might be limiting the database as well. My forum runs off an NFS share for the database drives but they are pretty high I/O. An SSD would of course be huge.
|
__________________
One man's reason that something is not reliable evidence is another man's whine about how others won't buy 3 magic beans with the family cow. - hgc |
|
3rd August 2014, 01:03 PM | #164 |
Technical Admin
Join Date: Jul 2006
Location: Canada's Texas
Posts: 1,495
|
I mentioned this already to you but I thought I'd bring it up here as well as a more general discussion.
There's two ways to do it, one is to purchase servers and colocate them, the other is to just rent servers. Renting costs more in the long run in general, but you have the not insignificant benefits of being supported by someone else, hardware repairs being someone else's problem, warranty being someone else's problem, and what to do if moving to updated hardware being someone else's problem.. well you get the idea Paying monthly for servers is just so much simpler IMO. Purchasing servers costs more up front, then has lower cost for a while (assuming the colocation is cheaper than renting the server), then increases again when things start to break or require purchasing more warranty or paying hands and eyes at the colo to do repairs. I prefer renting, but that kind of goes against the donation style funding. Not saying donations shouldn't be done, or part of it, or be the whole funding method, just raising the point for consideration. |
__________________
One man's reason that something is not reliable evidence is another man's whine about how others won't buy 3 magic beans with the family cow. - hgc |
|
3rd August 2014, 01:08 PM | #165 |
Technical Admin
Join Date: Jul 2006
Location: Canada's Texas
Posts: 1,495
|
Just a comment on this, while I LOVE Percona and have used their products in the past, this won't work for the current forum without alterations, as vBulletin 3.x uses MyISAM table types (and that's for InnoDB or XtraDB).
That said I have seen some vBulletin 3.x forums alter their table types to InnoDB, so that might be an option if that was done. |
__________________
One man's reason that something is not reliable evidence is another man's whine about how others won't buy 3 magic beans with the family cow. - hgc |
|
3rd August 2014, 01:27 PM | #166 |
Just the right amount of cowbell
Join Date: Oct 2008
Location: Well past Hither, looking for Yon
Posts: 6,710
|
Do we know what else is running on the hardware? I've seen antivirus programs that would use >90% of the system resources for half an hour or more, at random intervals (and that was considered a feature!), rendering an otherwise fast machine completely useless for even things like word processing.
Could we have some rogue mundane process occasionally using all the CPU, or allocating so much RAM that the Forum can't get any and has to resort to disk thrashing? |
__________________
"In times of war, we need warriors. But this isn't a war." - Phil Plaitt |
|
3rd August 2014, 01:45 PM | #167 |
Technical Admin
Join Date: Jul 2006
Location: Canada's Texas
Posts: 1,495
|
One last post, I missed a bunch of posts so I apologize for all the replies in order.
So some general my thoughts about the various areas: Ownership: Be it buying servers via donations, or holding donations to pay for monthly services, the question of who owns what and how they're accountable and such is something to consider. No one wants a situation where someone gets mad and takes their servers and goes home. Not a pleasant thing to think about something that warrants some thought. Hardware Source: As mentioned you can rent servers or buy and colocate them, so which path will need to be decided. Reliability: In general (and users will of course disagree) I don't think the forum is a mission critical service requiring five nines of availability. So spending a lot of money on redundancy probably isn't necessary; depending on the service provider's or vendor's warranty and reasonable levels of backups is probably fine. A day of downtime while a server's power supply is replaced or the database drive is replaced and restored from backup is acceptable for the forum IMO. Hardware: I'd suggest two servers, one for the database and one for the web server. An SSD for the database is a good idea, spindle drive for OS and database logs and SSD for database files. The web server doesn't require as much in the way of disk I/O, so two mirrored drives is sufficient IMO. OS: CentOS is fine to use, lots of support and info out there. I think CentOS 7 is even available now. Web Server: Apache works well and I'm not sure nginx is any faster for PHP performance. Static file performance is higher, but I think we should be offloading static resources to a CDN anyway, so Apache is probably fine, but nginx might be fun to use anyway (haven't played with it personally). Database Server: As mentioned previously, vBulletin 3.x uses MyISAM table types, in which case I'd suggest MariaDB as the database server. It's a drop-in replacement for MySQL but has other things that MySQL either doesn't have, is coming, or is exclusive to the enterprise version. If we converted the table types to InnoDB then we could use Percona which is also a drop in replacement for MySQL. I prefer Percona but maybe that's just because I've been reading the mysqlperformance blog for a long time. CDN: As mentioned using a CDN like MaxCDN can reduce the load on the HTTP server a lot (most of the requests in a vBulletin page are for static resources). Then the HTTP server can focus on serving up PHP pages. It's relatively cheap and there are a few ways to set it up, someone really skilled with rewrite rules can do it completely via rewrite rules. Personally I just changed the file locations in the vBulletin software itself; seemed easier to me. Images: Related to that is images in general. These forums allow for images to be uploaded, and icerat mentioned that the forum DB is 32GB which seems quite high to me (my forum is 4.2GB for 5 million posts). If images are stored in the DB then yes that's a significant performance issue, and a storage issue too. In my opinion there are a lot of different free image hosts out there and this is a forum, not an image server. On my forum we simply don't allow image uploads other than avatars, if someone wants to show an image they can upload it to imgur.com themselves. But at bare minimum images should be on the filesystem not in the database, and the CDN could also be used to offload the request for images from the server. Backups: Technical issues on how to backup the database aside, something like Crashplan ($10/mo business) is inexpensive IMO and gets you offsite backups as well (though Crashplan for Linux is a more complex setup). Some level of backups might also be included with whatever server or colocation plan is obtained. All I can think of at the moment. |
__________________
One man's reason that something is not reliable evidence is another man's whine about how others won't buy 3 magic beans with the family cow. - hgc |
|
3rd August 2014, 01:47 PM | #168 |
Technical Admin
Join Date: Jul 2006
Location: Canada's Texas
Posts: 1,495
|
I don't think anyone has any access to the server as of yet, the stuff icerat posted was forwarded to him if I understand correctly.
So yeah could be something else on the server, heck could be a hacker running a botnet or mining bitcoins! I doubt the server's been compromised but until one looks one never knows. |
__________________
One man's reason that something is not reliable evidence is another man's whine about how others won't buy 3 magic beans with the family cow. - hgc |
|
3rd August 2014, 03:36 PM | #169 |
NWO Cyborg 5960x (subversion VPUNPCKHQDQ)
Join Date: Feb 2008
Location: Starship Wanderer - DS9
Posts: 14,282
|
|
__________________
ModBorg Engine: Ibalgin 400 |
|
3rd August 2014, 07:09 PM | #170 |
Observer of Phenomena
Pronouns: he/him Join Date: Feb 2005
Location: Ngunnawal Country
Posts: 87,212
|
I'd just like to say something. A lot of people are posting in this thread, contributing ideas and thoughts. Sharon has asked for people who might be able to help, and that's all very well, but we don't want a huge number of people on the committee unless we want to recreate the forum in the form of a camel.
It seems that everyone feels entitled to input on the discussion. A project like this cannot work that way. A small lean team will be able to get the project completed faster and with fewer complications. This does of course mean that some people are going to feel ignored. I think we as a community need to get over that and provide only the help that is asked for, when it is asked for. Let's let those who are working work, and not expect them to take every possible opinion into account. |
__________________
So take that quantum equation and recalculate the wave by a factor of hoopty doo! The answer is not my problem, it's yours. Three Word Story Wisdom |
|
4th August 2014, 05:10 AM | #171 |
Philosopher
Join Date: Apr 2008
Location: sweden
Posts: 5,764
|
Personally I'm quite happy for people to give whatever their thoughts or ideas are and discuss them here.
Techinical issues are but one area though. From what I understand the general feeling withing JREF is that they would like the forum to be wholly independent of JREF. I don't specifically know why but given the number of lawsuits and threats that have been flying around the skeptic community on various issues of late, I can understand why there might be motivation. Options for doing this while still remaining affiliated (and keeping the name) are being considered. The question arises as to the organisational form of any future body operating the forum. A new non-profit? Part of an existing non-profit or for-profit entity? Couple of dudes in their mother's basement? None of the above? Just FYI, the current advertising income does not come close to covering the costs of the forum. While an upgrade to better equipment will, paradoxically, probably result in lower costs, it will likely still not be enough so ongoing funding solutions will need to be found. Throw in your 2 cents or relevant currency. |
__________________
Benford's law of controversy - Passion is inversely proportional to the amount of real information available |
|
4th August 2014, 05:31 AM | #172 |
Nitpicking dilettante
Administrator Join Date: Mar 2007
Location: Berkshire, mostly
Posts: 57,668
|
I'd rather wait and see what the JREF's position regarding the forum is, before making plans to go solo. As I understand it, TAM would not have happened without the forum. Certainly I would not have heard of TAM otherwise, nor attended half a dozen. The forum provides the volunteers and TAMbassadors that help run TAM, as well as providing scholarships for those unable to afford the cost. Of course, some of could still happen without the forum, but I think participation would be lower.
|
__________________
The whole problem with the world is that fools and fanatics are always so certain of themselves, and wiser people so full of doubts.Bertrand Russell Zooterkin is correct Darat Nerd! Hokulele Join the JREF Folders ! Team 13232 Ezekiel 23:20 |
|
4th August 2014, 05:43 AM | #173 |
Time Person of the Year, 2006
Join Date: Apr 2005
Location: Right here!
Posts: 19,246
|
|
__________________
I've always believed that cluelessness evolved as an adaptation to allow the truly appalling to live with themselves. - G. B. Trudeau A person is smart. People are dumb, panicky dangerous animals and you know it. - Kay, Men in Black. Enjoy every sandwich. - Warren Zevon |
|
4th August 2014, 05:51 AM | #174 |
Philosopher
Join Date: Apr 2008
Location: sweden
Posts: 5,764
|
That would be this Mr Adams for those interested. So as you can imagine he has the technical nous but perhaps has other interests he prioritises over this forum
Quote:
40% of queries from search engine bots sounds a bit high (around 30% is more typical) but certainly within normal range. |
__________________
Benford's law of controversy - Passion is inversely proportional to the amount of real information available |
|
4th August 2014, 06:02 AM | #175 |
NWO Cyborg 5960x (subversion VPUNPCKHQDQ)
Join Date: Feb 2008
Location: Starship Wanderer - DS9
Posts: 14,282
|
Right now, Sucuri proxy is causing problems.
http://www.internationalskeptics.com...1&postcount=11 |
__________________
ModBorg Engine: Ibalgin 400 |
|
4th August 2014, 11:00 AM | #176 |
Gatekeeper of The Left
Join Date: Sep 2007
Location: The Universe 35.2 ms ahead of this one.
Posts: 37,538
|
If systems help is needed, consider me a volunteer.
|
__________________
For what doth it profit a man, to fix one bug, but crash the system? |
|
4th August 2014, 12:18 PM | #177 |
Penultimate Amazing
Join Date: Oct 2002
Posts: 28,766
|
|
4th August 2014, 12:43 PM | #178 |
Philosopher
Join Date: Apr 2008
Location: sweden
Posts: 5,764
|
From my current understanding of the system (and I don't have all the facts) I think we could get a way more than good enough dedicated server or VPS for the $250 to $300/mth range.
We'd then want to add some firewalling and backup, maybe another $200. So we're talking max $500/mth for a better set up than now for hardware/hosting at less cost (which I believe is $500/mth+ not counting firewalling/backup?) The current vbulletin licence is privately owned, no ongoing costs, but I tend toward upgrading to Xenforo ($250 incl enhanced search) plus a yearly fee for ongoing upgrades/support ($65). While there's quite a lot of expertise available here I think it's worth paying for some migration software and an experienced tech to handle it. Maybe $500. (ironically some of that is licencing fees to Brian Dunning's partner in crime Shawn Hogan, but that's another story) So I'd say we're looking at say $1500 for setup and first month, then max $5-600/mth ongoing. Maintenance is currently all volunteer. Experience to date suggests that might need to change. Personally I'm in favor of pro-moderators/admins on a site this well trafficked. I don't have enough info on the workload to judge costs at this stage. Right now there is very limited advertising on the site that brings in little revenue. A mix of appropriate advertising plus member donations/subscriptions should be enough to support the site. My preference - and others I've spoken to seem to feel the same - is that a new non-profit spun off from JREF is probably the way to go. (any legal/accounting folk with time/expertise in setting up and maintaining a non-profit, put your hand up) I'm also getting the distinct impression there's a lot of backroom history we're dealing with here which has lead to the current somewhat critical situation. That may also make this entire discussion moot and something entirely different happens. Left hand, meet right hand, pleased to meet you. |
__________________
Benford's law of controversy - Passion is inversely proportional to the amount of real information available |
|
4th August 2014, 01:54 PM | #179 |
Time Person of the Year, 2006
Join Date: Apr 2005
Location: Right here!
Posts: 19,246
|
Somewhat rambling thoughts:
I've worked with a lot of non-profit organizations and have been president of several. While $1500 startup and $6000-$7000 a year does not sound like a lot, it can be to a huge amount to non-profit. I agree with your point about a pro-admin/mod being needed. Volunteers are great, but their priorities can shift so needed activities don't happen when needed, they get done when they can be done. So you're talking more money. I don't know how much time would be needed for a pro-admin and therefore how much money. I'll throw out a number of $20K per year. I don't know if that's too much or more likely too little. Will the community here support about $2000+ a month? Advertising can bring in revenue, but how much? I believe we have about 1500 frequent users. Could we get 150 to subscribe at $10-$15 a month? 10% participation in something like that is probably optimistic. In non-profits it comes down to a matter of whether or not something can be done financially, not whether or not it should be done. If you're spinning it off to a new non-profit, that can be done for $1000 or less, but would JREF want that? |
__________________
I've always believed that cluelessness evolved as an adaptation to allow the truly appalling to live with themselves. - G. B. Trudeau A person is smart. People are dumb, panicky dangerous animals and you know it. - Kay, Men in Black. Enjoy every sandwich. - Warren Zevon |
|
4th August 2014, 02:13 PM | #180 |
Gentleman of leisure
Tagger
Join Date: May 2005
Location: Flying around in the sky
Posts: 28,092
|
Making the forum independent of JREF would have huge consequences. Like loss of control of what happens on the forum. Maybe rule 8 could go out the door. Want to sue JREF while still an active member here and boast about it on the forum? Fine. Go ahead.
|
__________________
This signature is for rent. |
|
4th August 2014, 02:16 PM | #181 |
Philosopher
Join Date: Jun 2002
Posts: 5,012
|
The place is running like a dream. No real delays.
Some-one needs a lot of thanks for their work behind the scenes |
4th August 2014, 02:22 PM | #182 |
Guest
Join Date: Mar 2004
Posts: 21,840
|
Yes, it's running quicker, but many people still can't log in and I can't access the registration queue, which is growing and growing and growing:
|
4th August 2014, 02:22 PM | #183 |
post-pre-born
Join Date: Dec 2003
Location: Santa Barbara, CA
Posts: 25,183
|
I would very much like not to see this happen. I know nothing of the backroom but I still am of the opinion that there is a mutually beneficial relationship between JREF and this forum. TAM, as noted, is just one example. I suspect the forum acts as a feed for new JREF members and that that is a two-way street. The spin-off of Skeptical Community is an example to avoid.
|
4th August 2014, 02:24 PM | #184 |
Time Person of the Year, 2006
Join Date: Apr 2005
Location: Right here!
Posts: 19,246
|
|
__________________
I've always believed that cluelessness evolved as an adaptation to allow the truly appalling to live with themselves. - G. B. Trudeau A person is smart. People are dumb, panicky dangerous animals and you know it. - Kay, Men in Black. Enjoy every sandwich. - Warren Zevon |
|
4th August 2014, 02:36 PM | #185 |
Philosopher
Join Date: Apr 2008
Location: sweden
Posts: 5,764
|
Yes, been involved in several myself. As I mentioned earlier I'm a fan of developing business systems that allow community organizations to be self-funding and even profitable. The challenge is finding ways to do that. In the US there's actually a formal structure for this now - B-Corporations.
Quote:
Quote:
It has to be done in a way that is acceptable and useful to the community in question. It's also more work, but generates more revenue than things like googleads. As an example, if this was an independent community it may take a small commission on TAM ticket sales that occur via the site. I'm sure there are other skeptic events around the world that could do with co-marketing through this site.
Quote:
|
__________________
Benford's law of controversy - Passion is inversely proportional to the amount of real information available |
|
4th August 2014, 02:44 PM | #186 |
Banned
Join Date: Jun 2006
Location: Ivory Tower
Posts: 20,632
|
Like I suspected and icerat confirmed:
I'd advise to focus on first fixing the immediate problems and pressuring the host a bit (that specs are way above what I expected, there could be simply some defect stuff involved as it was the last time with that router/switch that wasn't fixed for ages) before getting into some pipedreams. I don't doubt that at this stage some serious money could be raised from frustrated forum members, but in the long run it will be difficult to sustain such high running costs including however enthusiastic professional helpers. Oh, and it seems to be much faster already indeed. |
4th August 2014, 03:52 PM | #187 |
Muse
Join Date: Nov 2004
Location: Santa Barbara, CA
Posts: 686
|
This is a very constructive discussion. I can't contribute anything to the technical side, but a brief comment about money might be helpful.
We have a detailed estimate from icerat that things could be made to run quite well for $500~600 a month. To be way generous, call it $10,000 a year. According to JREF's 2012 990 form, annual JREF revenue is around $1.3 million. Now there are lots of calls on that revenue, but it seems the forum could be kept running for about 1/10th of 1 percent of the JREF budget. This suggests to me the issue is JREF's level of interest, rather than any real money problem. It would be good to have a frank answer from JREF before worrying too much about funding models. |
4th August 2014, 04:26 PM | #188 |
Graduate Poster
Join Date: Sep 2006
Location: Van Squad
Posts: 1,484
|
|
4th August 2014, 05:42 PM | #189 |
beer-swilling semiliterate
Join Date: Jul 2006
Location: Connecticut, or King Arthur's Court. Hard to tell sometimes.
Posts: 25,791
|
I think you're off one decimal place; it would be about 1% of JREF's revenue. Still and all, it's not a huge amount of money, even for a comparatively small non-profit such as JREF.
Could the forum be self-sustaining? Possibly; if the membership put up, say, the hardware cost plus the first year's ongoing expenses, there would be time to develop some sort of model to generate revenue (I hate banner ads like the plague, but I would suffer them to preserve the forum, and I think most here would agree). |
4th August 2014, 06:01 PM | #190 |
Penultimate Amazing
Join Date: Feb 2010
Posts: 10,293
|
With the number of forum grants to TAM given out each year (20+ this year, for example), I'm quite sure that the forum can probably be self-sustaining just from the income that brings the JREF. I'm hoping they are taking that into account--or maybe they think they can raise more money elsewhere with equal or less effort. I'd be skeptical of that. That's another reason I agree with those that think it's likely more than just a money issue, but in any case the money issue will need to be dealt with eventually if we do want the forum to continue. |
4th August 2014, 07:09 PM | #191 |
Muse
Join Date: Nov 2004
Location: Santa Barbara, CA
Posts: 686
|
|
4th August 2014, 08:49 PM | #192 |
Technical Admin
Join Date: Jul 2006
Location: Canada's Texas
Posts: 1,495
|
My forum went from a single server to two servers at around the 6-700 concurrent users level (I suspect my forum usage profile has heavier peaks than this one though), so while yeah I was surprised the hardware was more robust than I had thought it was, I still wouldn't be shocked if the slowness was just server scaling.
And even if it is something specific that's eating up resources away from what should get the resources, I still think it's worth moving since you can likely get better servers for cheaper. But that's still a good point, taking a look at the server and making sure there isn't something else going on should be one of the first things done. vBulletin itself has some cron type jobs it runs on various intervals. I had an issue once where someone's browser half crashed in a way that was re-submitting the same request to the server many times a second, basically DoSing the server. I figured the user from the IP and emailed them and they rebooted their computer and it stopped. Strange stuff can happen. But granting volunteers root access to the server is something I can see them being reluctant to do, that's a pretty big amount of trust. |
__________________
One man's reason that something is not reliable evidence is another man's whine about how others won't buy 3 magic beans with the family cow. - hgc |
|
4th August 2014, 10:49 PM | #193 |
Join Date: Sep 2007
Posts: 10,924
|
If the hardware fixes the problems please do not do this. The forum works great (see below post). Do not fix what is not broken. ETA: And do not overfix what is broken. Like how some computer repair shops actually fix problems, even if it takes some time. Other computer repair shops just instantly reformat and reinstall. |
__________________
________________________ |
|
4th August 2014, 10:57 PM | #194 |
Join Date: Sep 2007
Posts: 10,924
|
I have to agree with Dan here. The forum works absolutely fantastic. Lightning fast page loads. For hours on end. And then randomly it will work horribly slow for hours. On and off, just like that, for at least 1 year now. Does not compute. My first guess would be the large volume of traffic from China and search engines, but it could be some random glitch not yet troubleshot. |
__________________
________________________ |
|
4th August 2014, 11:07 PM | #195 |
Philosopher
Join Date: May 2007
Posts: 6,900
|
Yes - the suggestion of a rogue process, sitting in memory and butting in on everything else, that many have made makes sense of the vmstats (the over 200 X as many interrupts as normal, only half as many processes running, free memory shrinking, even before things start to back up midway through the snapshot); certainly worth looking into.
|
__________________
"Say to them, 'I am Nobody!'" -- Ulysses to the Cyclops "Never mind. I can't read." -- Hokulele to the Easter Bunny |
|
4th August 2014, 11:55 PM | #196 |
post-pre-born
Join Date: Dec 2003
Location: Santa Barbara, CA
Posts: 25,183
|
I'd be a bit leery of such an approach. What happens after year one if a revenue model is not found? Is there some way to estimate revenue streams from various business models before starting down one road? Surely, yes, because we can't be the first non-profit to fact this situation.
|
5th August 2014, 01:08 AM | #197 |
Chief Solipsistic
Autosycophant Join Date: Jan 2007
Location: Vancouver, Canada
Posts: 13,414
|
I've got a problem...I can view the forum no problem, but every time I try to make a post, I'm taken to the Sucuri Website Firewall, with the following information:
Block details Your IP: 123.151.32.96 URL: forums.randi.org/newthread.php?do=postthread&f=26 Your Browser: Mozilla/5.0 (Windows NT 6.1; rv:31.0) Gecko/20100101 Firefox/31.0 Block ID: BLACK02 Block reason: Your IP address is listed in our blacklist and blocked from completing this request. Time: Tue, 05 Aug 2014 02:35:34 -0400 Server ID: cp413 I've had issues with the Sucuri thing before, but never this error message. It seems to arise only when I'm making posts, not when I'm just viewing the forum. Given the previous information that a lot of traffic is coming from China, I'm concerned that perhaps someone changed the security to block Chinese traffic? I'm currently accessing the forums through a VPN, but that's far from an ideal solution. |
__________________
Please check out my business, The Language of Culture |
|
5th August 2014, 01:39 AM | #198 |
Banned
Join Date: May 2012
Location: Belgium
Posts: 3,398
|
Sharon (idoubtit), I hope Mr. Adams, who is perhaps the person who's been installing the Sucuri website firewall (see here for an explanation) for the forum in recent days, is fully aware of the problems mentioned by Lisa Simpson and Wolfman above. The number of active users is currently rather low (about 325 now, usually about 700 [?]), this may explain why the forum is fast. It would be nice if you could convince this person (or whoever is currently in charge of the hardware) to post in the Forum Help & Member Support sub-forum. In this way, he could explain what he's doing, interact with, and receive advice from the experts of this forum.
|
5th August 2014, 02:01 AM | #199 |
NWO Cyborg 5960x (subversion VPUNPCKHQDQ)
Join Date: Feb 2008
Location: Starship Wanderer - DS9
Posts: 14,282
|
|
__________________
ModBorg Engine: Ibalgin 400 |
|
5th August 2014, 02:45 AM | #200 |
Thinker
Join Date: Mar 2008
Posts: 169
|
The last few columns of CPU stats suggest there's plenty of CPU time left. The "wa" column is relatively high, indicating that the OS has to wait for I/O (probably disk, could be network too), but it's nothing problematic. System cpu usage is really low and most work is done in user space, indicating that the application software isn't eating unnecessary kernel resources. To finish there's plenty of CPU time left, so CPU is not the bottleneck for this snapshot. Context switches are fairly high but nothing extraordinary. I have systems that run way higher on all accounts and are still responsive.
It smells indeed like a software scaling problem, which is best solved in the application, but you could improve a lot with better hardware of course. I would concentrate on the disks first, no the CPU. Any chance you can take some "top" snapshots at peak load, or more vmstats, maybe that will show a different picture? If money allows, go indeed for SSD. Alternatively for disks don't use RAID5: RAID10 or RAID6 are much better for performance and resilience but require more disks. On the other hand RAM is cheap, you could cache most of the DB in memory, only leaving writes loading the disks. Upgrading from CentOS 5 to 6 might give between 10 to 30% more performance on the same hardware, contributed from different parts of the OS. CentOS 6 supports the ext4 filesystem, CentOS 7 (no personal experience yet) uses XFS by default. Both will give some improvement for MySQL file system performance. The kernel and networking stack will do the rest. Even Upgrading to the latest CentOS version can already help, I've seen about 10% difference in CPU load on a heavily loaded server when upgrading from 5.4 to 5.10. You should also checkout ktune and tuned, especially on CentOS 6 this might give a free additional benefit. |
Thread Tools | |
|
|