So I'm working on a website uptime monitor that should check thousands of websites per minute by doing a simple http call to them and checking the received status code.  I've tested it using multiple processes of a node.js implementation to ensure that the bottleneck is not a programming or processing one.  Anyways I've hit a wall. The maximum number of sites I'm able to check is roughly 2000 per minute. This number doesn't change if I run 1 instance or 10 instances of the code.  If I run multiple processes, the throughput of each process reduces so that the total output is still 2000 sites checked per minute.  The actualy network bandwidth doesn't seem to be too high. I mean 2000 websites would mean roughly 1 mb of data (and that's being generous). So it's probably an issue somewhere else and i'm trying to pinpoint that.  Tested on digital ocean.  Thanks in advance.
3061
This is a  about capacity planning for web sites. Related:      What are some recommended tools and methods of capacity planning for web sites and web-applications? Please feel free to describe different tools and techniques for different web-servers, frameworks, etc., as well as best-practices that apply to web servers in general.
67014
We are running two production servers with Apache 2 and MySQL. I am looking for a reliable way of monitoring our load, stability and uptime.  I have come across , but are there better alternatives?
115571
Ideally, I'd like to hear of an application to show how much bandwidth each application is consuming on my local Windows PC. Alternatively, instructions to use any other application for this purpose. The simpler the better. Something that works for Windows 2000/XP/Vista/7 (if possible).  Any ideas? Thanks.
99674
This is a situation I am frequently in:   I have a source server with a 320GB hard-drive inside of it, and 16GB of ram (, but as this is an issue I run into frequently on other machines as well, I would prefer the answer to work on any "reasonable" Linux machine) I have a backup server with several terabytes of hard-drive space (, see disclaimer above)   I want to transfer 320GB of data from the source server to the target server (specifically, the data from /dev/sda).   The two computers are physically next to each other, so I can run cables between them. I'm on a LAN, and , which means my network speeds should "ideally" be 1000Mbit, right? Security is not an issue. I am on a local network, and I trust all machines on the network, including the router. (optional) I don't necessarily need a signed checksum of the data, but basic error checking (such as dropped packets, or the drive becoming unreadable) should be detected rather than just disappear into the output.     I searched for this question online, and have tested several commands. The one that appears the most often is this:  ssh user@192.168.1.100 'dd bs=16M if=/dev/sda | gzip' &gt; backup_sda.gz   This command has proven too slow (it ran for an hour, only got about 80GB through the data). It took about 1 minute and 22 seconds for the 1GB test packet, and ended up being twice as fast when not compressed. The results may also have been skewed by the fact that the transferred file is less than the amount of RAM on the source system.  Moreover (and this was tested on 1GB test pieces), I'm getting issues if I use the gzip command and dd; the resulting file has a different checksum when extracted on the target, than it does if piped directly. I'm still trying to figure out why this is happening.
44956
Does anyone know?  And a bigger question is what happens when you encounter this maximum? Is this the same number with other Windows OSs such as Vista, XP etc.?
15330
I have a website that has about 100,000 nodes. In the core search page (admin/config/search/settings) it shows that:  There are 80801 items left to index.  It index 100 (or at most 500) nodes per cron run but it's useless. I want to bulk index the whole site (all nodes) for core search. How can I do that? The  and  modules are created for this but there are only a few users use these modules and I couldn't be sure to use them. And also there is a module called  for modifying the cron per job but it's still useless for me because there are thousands of nodes to index. (Note: I want to do this for core search not for other search solutions).
14660
What are the meaning of the TIME_WAIT when using netstat of my web server process? I am sure the web server is not over loaded.  tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      15655/apache2 tcp        0      0 x.x.x.x:80       123.125.66.35:19667     SYN_RECV    - tcp        0      0 x.x.x.x:80       113.138.59.140:12186    TIME_WAIT   - tcp        0      0 x.x.x.x:80       66.249.67.121:59493     ESTABLISHED 23702/apache2 tcp        0      0 x.x.x.x:80       69.28.51.206:40652      TIME_WAIT   - tcp        0      0 x.x.x.x:80       221.126.149.99:51877    TIME_WAIT   - tcp        0      0 x.x.x.x:80       221.126.149.99:51872    TIME_WAIT   - tcp        0      0 x.x.x.x:80       123.125.66.19:13084     TIME_WAIT   -
3914
I am writing a server, and I send each action of into a separate thread when the request is received. I do this because almost every request makes a database query. I am using a threadpool library to cut down on construction/destruction of threads. My question is: what is a good cutoff point for I/O threads like these? I know it would just be a rough estimate, but are we talking hundreds? Thousands? How would I go about figuring out what this cutoff would be?  EDIT: Thank you all for your responses, it seems like I am just going to have to test it to find out my thread count ceiling. The question is though: how do I know I've hit that ceiling? What exactly should I measure?
51953
When I plug my laptop via LAN cable and check speed on speedtest.net I have over 32 Mb/sec. When I unplug the cable and connect to the same router I have less than 15 Mb/sec. I tried to change channels already but this didn't help. I'm sitting just near the router.   What can it be? The router is a Dlink 400.
41754
I'm a programmer but I don't know much about stats, so please excuse me if this question sounds naive.  I have two computers that are in different locations.  I have made N measurements for the network distance between them.  So it's essentially a series of N numbers like 200ms, 173ms, 212ms....  Now, I need to write a simulator that simulates these two computers.  What I need is a generator that can generate new network distances between them.  So for instance, if the generator generates something like 183ms, then it'd be reasonable.  But if it generates, say, 1 billion ms, then that wouldn't make sense since it deviates so much from the original data set.  So how would I generate new numbers that are similar to the numbers in the original series?  P.S. I'm not sure what tags I should use for this question because I don't know much about stats... feel free to add tags that you think are more appropriate.
46452
I'm going to buy a new router, but, I've just realised that for nearly all routers I've looked at (Belkin, Netgear, D-Link) don't state the maximum number of devices which can connect concurrently.   I have a  and have now looked on their website, and through the manual and also find no mention of the limit (which I really need to know if there is one due to debugging another issue).   The new router will be a gift for a friend... The issue is, they have 12 devices in their house, all of which will require WiFi and another 3 devices which are to be hard wired.  Since the router websites don't mention any limitations, can I assume there is a general limit on what the number of concurrent connected devices can do or is the issue more about understanding the more devices connected, the less the performance will be due to the sharing of the resource?  My question is, without trying to determine the limit my self manually (plugging in devices until it's maxed out), is there any way to know that a certain device will work with a given number of concurrent connected devices?
68311
The connection speed our webapp is showing (from my office) is much slower than my connection speed to a SpeedTest.net site in the same city.  I can test my internet using SpeedTest.net to Houston (where our server is): 37 Mb/s And we have a built in test in our app which shows 3.5 Mb/s  I'd like to test both "houston" and $ourDomain with the SAME service. But all I can find are sites that test the actual page, which I assume introduces a lot of extra variables.
110458
I'm working in marketing cloud project. We need to use a lot of API calls because we have so much real time use cases.   I would like to know if there is any limit in the number of calls per day or per hour ?   If yes, which is the max number of API calls ?  Any help will be so much appreciated.  Thank you !
62855
I have been peaking in macOS Activity Monitor and found this:  One tab spawns atleast 3 processes:   Firefox ~400MB FirefoxCP Web Content ~200MB FirefoxCP Web Extensions ~200MB     Update - ran again and got these numbers ...
123335
Right now I'm in the process of possibly switching from a Cable provider to a DSL provider.  I  have both connections live, and before I cancel one or the other, I'm wanting to do some exhaustive testing of the internet connection.  I have three major questions:   What are some approaches that I can quantitatively test the speeds (both up and down) and quality of my internet connections (ping, time connection is down, etc,.)?   Are there other consideration that should be taken when testing an internet connection?   Are there any tools that can do this automatically and capture results?   Note: if you decide to suggest a piece of software, please  in doing so.  Also I'm looking to be able to save any data from the software for detailed analysis and comparison  Overall, I'm looking to compare the two connections over multiple periods of time such as peak hours (1600 - 2100 in my area), and with different loads such as streaming movies, uploading files, etc,.         This question was Featured in the blog!    Read the  for more details or  yourself
58583
For the past several years, I have been using a simple software bandwidth monitor on my computer to keep track of my monthly bandwidth usage. It has worked fine (more or less; the numbers are slightly off from my ISP’s online tracker).  This works fine for a single computer, but when my mother connects her laptop to my router or if I use  second computer, then monitor and managing bandwidth becomes much more difficult.  I considered running a bandwidth monitoring software on each system that is to connected and (somehow) transferring the logs to a central system to (again somehow) be incorporated into a central log file, but not surprisingly, this is a nightmare at best.  I also thought about a router solution (as has been mentioned, but it would need to be easy to use and customize).  Is there an easy way to monitor and log total bandwidth usage (ie from multiple systems)? It should not require all the systems to be monitored to be on at any specific time, nor having all systems connect through another one (i.e., they should be able to connect directly to the Internet connection).  Also, it should be able to differentiate between local and remote traffic so that transferring files between two local systems does not count towards Internet bandwidth usage.
12311
CPU Cycles, Memory Usage, Execution Time, etc.?  Added: Is there a quantitative way of testing performance in JavaScript besides just perception of how fast the code runs?