Hello all, anyone have a suggestion for how to achieve the effect of directly injecting a path to a specific peers adj-out? For example given the following setup:
-
GoBGP server has N peer groups of same AS named (e.g. "ams", "iad", "phx")
-
Each peer group has N neighbors, usually 2 (e.g. "ams01", "ams02")
-
Each peer group has a distinct next hop for a given prefix, example adj-in:
- ams01 adj-in: 50.8.0.0/16 10.20.30.30 ...communities...
- ams02 adj-in: 50.8.0.0/16 10.20.30.30 ...communities...
- iad01 adj-in: 50.8.0.0/16 10.20.30.31 ...communities...
- iad02 adj-in: 50.8.0.0/16 10.20.30.31 ...communities...
- phx01 adj-in: 50.8.0.0/16 10.20.30.32 ...communities...
- phx02 adj-in: 50.8.0.0/16 10.20.30.32 ...communities...
Given this, use the GoBGP API to advertise a more specific prefix with the same next hop and one or more new communities, resulting in the following adj-out:
- ams01 adj-out: 50.8.0.0/24 10.20.30.30 ...communities... ...newcommunities1...
- ams02 adj-out: 50.8.0.0/24 10.20.30.30 ...communities... ...newcommunities1...
- iad01 adj-out: 50.8.0.0/24 10.20.30.31 ...communities... ...newcommunities2...
- iad02 adj-out: 50.8.0.0/24 10.20.30.31 ...communities... ...newcommunities2...
- phx01 adj-out: 50.8.0.0/24 10.20.30.32 ...communities... ...newcommunities3...
- phx02 adj-out: 50.8.0.0/24 10.20.30.32 ...communities... ...newcommunities3...
First I tried simply advertising each path into the global rib with the same prefix but different communities which were matched to each peer group in the export policy. But I realized that each advertisement would cause an implicit withdraw of the previous path. I tried setting NoImplicitWithdraw = true, and thought it was exactly what I needed as the global rib contained the 3 distinct paths each with a separate set of communities to match each peer groups export policy, i.e. global rib had:
- global rib: 50.8.0.0/24 10.20.30.30 ...communities... ...newcommunities1...
- global rib: 50.8.0.0/24 10.20.30.31 ...communities... ...newcommunities2...
- global rib: 50.8.0.0/24 10.20.30.32 ...communities... ...newcommunities3...
But each time I added a new route, it would withdraw from the adj-out of the peer matching the specific set of communities. For example given an empty global rib, then advertising a single advertisement of:
- global rib: 50.8.0.0/24 10.20.30.30 ...communities... ...newcommunities1... ^ last advertisement
I would get the adj-out I wanted:
- ams01 adj-out: 50.8.0.0/24 10.20.30.30 ...communities... ...newcommunities1...
- ams02 adj-out: 50.8.0.0/24 10.20.30.30 ...communities... ...newcommunities1...
But as soon as I sent the next prefix to the gobal rib:
- global rib: 50.8.0.0/24 10.20.30.30 ...communities... ...newcommunities1...
- global rib: 50.8.0.0/24 10.20.30.31 ...communities... ...newcommunities2... ^ last advertisement
The adj-out for the previous paths would withdraw, leaving a new adj-out of:
- ams01 adj-out: 50.8.0.0/24 10.20.30.30 ...(MISSING: here to show desired result)
- ams02 adj-out: 50.8.0.0/24 10.20.30.30 ...(MISSING: here to show desired result)
- iad01 adj-out: 50.8.0.0/24 10.20.30.31 ...communities... ...newcommunities2...
- iad02 adj-out: 50.8.0.0/24 10.20.30.31 ...communities... ...newcommunities2...
Finally the last route would be the route that was exported to a neighbor:
- global rib: 50.8.0.0/24 10.20.30.30 ...communities... ...newcommunities1...
- global rib: 50.8.0.0/24 10.20.30.31 ...communities... ...newcommunities2...
- global rib: 50.8.0.0/24 10.20.30.32 ...communities... ...newcommunities3... ^ last advertisement
The adj-out for the previous paths would withdraw, leaving a new adj-out of:
- ams01 adj-out: 50.8.0.0/24 10.20.30.30 ...(MISSING: here to show desired result)
- ams02 adj-out: 50.8.0.0/24 10.20.30.30 ...(MISSING: here to show desired result)
- iad01 adj-out: 50.8.0.0/24 10.20.30.31 ...(MISSING: here to show desired result)
- iad02 adj-out: 50.8.0.0/24 10.20.30.31 ...(MISSING: here to show desired result)
- phx01 adj-out: 50.8.0.0/24 10.20.30.32 ...communities... ...newcommunities3...
- phx02 adj-out: 50.8.0.0/24 10.20.30.32 ...communities... ...newcommunities3...
Side note: The behavior above doesn't seem to send a WITHDRAW event when monitoring the rib. So to my local route caches it looked like the route was being accepted to each peer, since the previous route was never withdrawn from my local cache. It wasn't until I checked the actual peers routing tables directly that I noticed the route missing.
Since then I have tried many types of configurations and work arounds. Trying a route server mode looked promising because it seemed that each neighbor had its own distinct rib. But I can't find a way to inject directly into the local rib of a specific neighbor using the GRPC API as there is no "Name" param like the corresponding GetRib / GetPath requests.
For exmaple when selecting the next hop for the advertisements above I use the neighbors address as the "Name" in the GetRib / GetPath requests with a resource of Resource_ADJ_IN. However AddPath has no way to specify a Resource_ADJ_OUT (which sort of makes since, it would circumvent route policy, but also be very useful I think for strictly managed routing via API).
I have also tried VRF since it provides distinct routing tables, but I am not familiar with RT / RD / import / export aspects of it so getting a working POC was a struggle. Not knowing if it was the right tool for my problem made me punt exploring further.
Finally I've tried various combinations of local / remote AS since changing
the peer AS is an option. But I can't get paths to export at all in these
cases, my BGP knowledge may be the limiting factor here but I struggled to
actually set the source AS through the table API. I ended up crashing the
GoBGP server no matter how I attempted to add a bgp.NewAsPathAttribute
. When
using NewAsPath4..
it crashed with:
"Topic":"Table","msg":"AS4_PATH must be converted to AS_PATH"
But when using NewAsPath
it crashed with:
"Topic":"Table","msg":"AsPathParam must be converted to As4PathParam"
This is probably a gap in my BGP knowledge / path construction, my question is could the use case be solved with the right API usage and replacing community driven export policy with an "as-path" matching one?