Hacker News
points by temp0826 1 day ago | 0 comments
Hasn't stuff like this been solved with anycast and typical routing algorithms of old etc? Or is this solving a different problem? (This doesn't affect me but have been considering moving some stuff from aws/cloudflare to bunny)
[–] vbernat 1 day ago
Some hobbyists do anycast on their own, but this is more complex. See https://tylerobrien.dev/posts/2022/06/28/building-an-anycast... for example. This also require some advanced traffic engineering: what looks close for BGP may be on the other side of the planet. Relying on DNS only is more accessible.
[–] cortesoft 1 day ago
I worked for a CDN that ran using anycast for 10 years, and there are a LOT of gotchas and trade offs that make it complicated to use.

For one, your ability to control traffic flow is limited to very coarse knobs, since where your traffic lands is dependent entirely on the BGP routes you advertise and who you advertise them to.

The basic idea of BGP is that you advertise the same IP blocks to multiple peers, and then every other network will choose the destination that is the least number of ‘hops’ away. However, hops are not always a perfect representation of the fastest, or even closest, destination. Sometimes a hop travels a far distance, or is over a slow or congested link. Sure, one connection might be 2 hops instead of 3, but if those two hops are slow the route with “more” hops might actually be faster.

You can solve this with a lot of different tricks, such as adding extra fake hops to your announcements in certain places, but you still have to be constantly monitoring the route traffic is taking to make sure your traffic continues to take the best route.

Secondly, there is no easy way to distribute traffic from a single network to multiple destinations for load balancing purposes. Every request coming from a particular network is going to make the same anycast calculation, so it will send all requests to the same destination. If you want to split that up into two different destinations, you have to go back to using DNS tricks or using different destinations IPs to split traffic up.

We would do this sort of load management by dividing traffic onto different IP blocks, and then dynamically advertising or not advertising those blocks from datacenters depending on load. If one datacenter is becoming overloaded, you pull some blocks from the advertisements for that datacenter, and that traffic stops being routed there.

There are a ton of other factors that go into it, but the bottom line is that anycast is not a simple fix to this problem.

[–] temp0826 1 day ago
I guess I'm just confused on who the actual target for this is (running your blog on 7 different servers around the world is uh...something...but I wouldn't call it practical/realistic). I wouldn't think that this type of thing is the first thing you reach for if your situation is actually calling for geobalancing (what level of the stack should this problem be solved, is maybe closer to what I'm asking).
[–] cortesoft 1 day ago
You are right, this is literally the exact use case for a CDN; serving static content around the world with low latency. You only need one origin server, or two for redundancy.
[–] vbernat 1 day ago
Of course, my setup is overkill. It's mostly something to play with.

But, I also serve HTML pages "myself." CDN is only used for assets. I want to control the integrity of the content. CSS and JavaScript files served by the CDN are protected by an integrity hash (SRI). If I could, I would also include images in this, but this is not possible (and not something that looks planned).

[–] PaoloBarbolini 1 day ago
Very few providers offer anycast. It's also kind of simpler to take servers from multiple providers and have DNS be the integration layer between all of those.
[–] dotwaffle 1 day ago
> Very few providers offer anycast.

Huh? Most of them offer anycast... Unless I'm misunderstanding what you're trying to say with this?

[–] PaoloBarbolini 1 day ago
What I meant to say is that if you're going to build your own CDN, or edge network or whatever, very few providers offer VMs or dedicated servers with anycast.