[system design] cache

Caching will enable you to make vastly better use of the resources you already have as well as making otherwise unattainable product requirements feasible.

The locality of reference principle: recently requested data is likely to be requested again.

Cache can hapen at all layers in the architecture, but tends to appear more closer to the front-end.

Placing a cache directly on a request layer node enables the local storage of response data. The cache can locate on both memory(very fast) and node’s local disk (faster than network storage)

Cache on multiple nodes: Each nodes can have its own cache. However if the load balancer distribute the requests to different nodes, the the other node might not know the cache exists in the original node. Thus we’ll need: global cache and distributed cache.

CDNs are a kind of cache that comes into play for sites serving large amounts of static media. In a typical CDN setup, a request will first ask the CDN for a piece of static media; the CDN will serve that content if it has it locally available. If it isn’t available, the CDN will query the back-end servers for the file, cache it locally, and serve it to the requesting user.

How does CDN work
a CDN is a network of servers linked together with the goal of delivering content as quickly, cheaply, reliably, and securely as possible. In order to improve speed and connectivity, a CDN will place servers at the exchange points between different networks. By having a connection to these high speed and highly interconnected locations, a CDN provider is able to reduce costs and transit times in high speed data delivery.

How does CDN reduce Latency

• Global distributed: reduce distance between users and website resources
• Hardware and software optimizations such as efficient load balancing and solid-state hard drives
• Reduce file sizes using tactics such as minification and file compression
• CDNs can also speed up sites which use TLS/SSL certificates by optimizing connection reuse and enabling TLS false start

If the data is modified in the database, it should be invalidated in the cache; if not, this can cause inconsistent application behavior.

• Write-through cache: Under this scheme, data is written into the cache and the corresponding database at the same time.
  Advantage
  1. We will have complete data consistency between the cache and the storage.
  2. Ensures that nothing will get lost in case of a crash, power failure, or other system disruptions.
     Disadvantage
  3. Higher latency for writing operations
• Write-around cache: Data is writen directly into permanent storage, bypassing the cache.
  Advantage
  1. Reduce the cache being flooded with write operations that will not subsequently be re-read
     Disadvantage
  2. A read request for recently written data will create a “cache miss” and must be read from slower back-end storage and experience higher latency
• Write-back cache: Data is written to cache alone and completion is immediately confirmed to the client.
  Advantage
  1. This results in low latency and high throughput for write-intensive applications
     Disadvantage
  2. This speed comes with the risk of data loss in case of a crash or other adverse event because the only copy of the written data is in the cache

1. FIFO (First In First Out)
2. LIFO (Last In First Out)
3. LRU (Discards the Least Recent Used First)
4. MRU (Discards, in contrast to LRU, the most recently used items first.)
5. LFU (Discards Least Frequently Used)
6. RR (Random Replacement) Randomly selects a candidate item and discards it to make space when necessary.