Caching Overview and Hardware

In this episode we give a general overview of caching, where it's used, why it's used, and what the differences in hardware implementations mean in terms we can understand. This will be foundational to understanding caching at a software level in an upcoming episode. There's also something about the number 37 that may be the most important number to remember...ever...

Podcast News

You can see all the show notes in their original form by visiting: http://www.codingblocks.net/episode45

Thanks for your patience, we had a couple of rough audio situations - and we appreciate you sticking with us!

iTunes Reviews Hedgehog, Thiagoramos.ai, Btn1992, Jonajonlee, UndeadCodemonkey, zmckinnon, hillsidecruzr, Dibjibjub, ddurose

Stitcher Reviews pchtsp, rafaelh, CK142, TheMiddleMan124, LocalJoost

• 60 TB SSD!!! https://www.engadget.com/2016/08/09/seagate-60tb-ssd/
• Samsung 950 Pro M.2 PCI SSD

  Samsung 950 PRO Series - 512GB PCIe NVMe http://amzn.to/2b0adZt Full Review http://www.anandtech.com/show/9702/samsung-950-pro-ssd-review-256gb-512gb

• Ready Player One (fiction book) http://amzn.to/2aPaCi3 Ready Player One - Amazing Audio Book

Clean Code episodes coming soon + book giveaway - Stay Tuned!

Caching: Turtles all the way down

Turtles all the way down??? https://en.wikipedia.org/wiki/Turtles_all_the_way_down

• Storing a subset of information for faster retrieval
• The hit ratio dramatically increases as the cache size increases
• Think about a simple web request…
• Browser cache
• DNS cache
• ISP caching
• CDN
• Whatever your application is doing (redis, framework, database, etc)
• PLUS whatever the various computers are doing

Why don't we cache everything?

• Fast is expensive!
• Cache Invalidation is hard!

Caching at the hardware level Interactive Cache Visualization

Latency Numbers Every Programmer Should Know https://gist.github.com/jboner/2841832

Relative Memory Access Interactive Demo http://www.overbyte.com.au/misc/Lesson3/CacheFun.html

Caching is a strategy that computers use going all the way down to the processor

L1

• .5ns
• As quick as a it gets, how long it takes light to travel 6"
• Managed by the CPU itself, no assembly available!

L2

• 7ns
• 14 x slower than L1
• L3 / L4 / Scratch etc

Main Memory

• Have numbers for a "reference" and a 1mb sequential read
• 100ns - 250,000ns
• 14 - 35,714 x slower than L2
• 200 - 500,000 x slower than L1

Network

• Sending is quick, there are numbers for that
• In general, a lot of variability here Same datacenter
• 500,000 ns
• 2 x slower than Main Memory
• 1 million times slower than L1

SSD

• Wait, network faster than the hd??? Yes, but no
• 1mb sequential
• 1 million ns
• 2 x slower than Network
• 2 million x slower than L1

Spinning Disk

• Get your employer to get you an ssd!
• 1mb sequential read
• 20 million ns
• 20 x slower than SSD
• 40 million x slower than L1

Internet

• Rough gauge of internet speeds
• Highly variable (CDN + ISP caching, for example), but gives you a sense of scale
• 150 million ns
• 7.5 x slower than spinning disk
• 300 million times slower than L1

In more relatable terms.

• 1 second for L1 Cache
• 5 days for memory
• 11 days for data center
• 23 days for SSD
• 15 months for HD
• Almost 10 years for internet!

Think about how those numbers cache

• RAM / Application cache
• Local Hard drive
• Network storage
• Cache Server
• DB

Summary

Hope we gave you a good idea of the importance and scale of caching in computing at the hardware level

Things we didn't talk about coming in a future episode:

• Application / Software caching and caching algorithms

Resources we Like

Latency Numbers Every Programmer Should Know https://gist.github.com/jboner/2841832

How L1 and L2 caching work http://www.extremetech.com/extreme/188776-how-l1-and-l2-cpu-caches-work-and-why-theyre-an-essential-part-of-modern-chips

Relative Memory Access Interactive Demo http://www.overbyte.com.au/misc/Lesson3/CacheFun.html

Miscellaneous

Michael's Favorite Meetup Ever The Atlanta JavaScript Meetup http://www.meetup.com/AtlantaJavaScript/events/222696324/?a=cr1_grp&rv=cr1

Hacking Interviews with: Nick Larsen - http://cultureofdevelopment.com/ Sam Lawrence - http://www.samelawrence.com/

Tip of the Week Algorithms to Live By

Joe: Algorithms to Live By There's something about the number 37%... http://amzn.to/2aX1iJk

Michael: Use Sublime to replace \n with an actual new line by turning on RegEx search and replace. Or in Michael's case, replace with actual \n\t characters. http://stackoverflow.com/questions/20515670/replace-n-with-actual-new-line-in-sublime-text

Allen: Collaborative Markdown Editor - What?! http://www.hackmd.io