This week in ABC: A closer look at High Bandwidth Memory, South Korean chipmaker SK hynix posts record earnings, and solar backpacks supply light after dark in Tanzania
Clear, well written! On comment: John von Neumann was not just a computer pioneer. He is considered one of the greatest mathematicians of modern history and made great contributions to fundamental physics as well.
So while inference operations generally are less memory intensive than training, a shift likely still means the total system memory requires more state-of-the-art HBM (more AI requests concurrently, latency demands, etc)
Clear, well written! On comment: John von Neumann was not just a computer pioneer. He is considered one of the greatest mathematicians of modern history and made great contributions to fundamental physics as well.
Absolutely!
If the AI workload shifts significantly from training to inference what effect does that have on memory demand ?
So while inference operations generally are less memory intensive than training, a shift likely still means the total system memory requires more state-of-the-art HBM (more AI requests concurrently, latency demands, etc)
Absolutely loved the read! Insightful!