... One common solution is to introduce a cache: before hitting the backend, you check your cache and use that value if present. This function is the single-key implementation of the Thundering Herd protection.
Re: Simple Solution To Thundering Train Herd Post by mrvn » Thu Feb 14, 2019 10:31 am The +1000 penalty per routed train is a nice idea too to balance trains between multiple stations. This means that when public figures start a live broadcast, we need to be able to handle the potential of more than a million people watching the broadcast at the same time, as happened recently with Vin Diesel’s live stream . When I talk about concurrent requests, what I really mean is near‑simultaneous requests that are coming in within your response window for that URL. cache_get_or_compute will attempt to fetch or compute the cached value for the given key, and will try really hard to avoid more than one user recomputing the cached value at any given time. What thundering herd really is, from my point of view, is when you’re getting so many requests within the window of your response time that your response time starts to slow. In computer science, the thundering herd problem occurs when a large number of processes or threads waiting for an event are awoken when that event occurs, but only one process is able to handle the event. When many readers simultaneously request the same data element, there can be a database read overload, sometimes called the “Thundering Herd” problem. This can happen under Unix when you have a number of processes that are waiting on a single event. When the processes wake up, they will each try to handle the event, but only one will win. The Thundering Herd Problem A note on the "thundering herd" problem: it's not really of much interest in this discussion. A Solution for thundering herd problem in node.js. Scheduler thrashing. This page addresses how to prevent it in a single JVM or a clustered configuration. The "problem" arises because people attempt to increase concurrency of accepting new connections by having multiple threads all blocking waiting on select(2). thundering herd problem. Solving the “thundering herd” problem Some public figures have millions of followers on Facebook.
When that event (a connection to the web server, say) happens, every process which could possibly handle the event is awakened. Another excellent resource to learning about how to solve The Thundering Problem in a much more complex scenario, I recommend this short video, from the Facebook Engineering team. This solution was introduced by Youtube (This video does not talk only about the Thundering herd problem, and it is a little bit outdated, but it is still relevant).