Archives

Closing plenary

28 October 2016

At 12 p.m.:



BENNO OVEREINDER: Welcome for the closing plenary, and this is chaired by myself and Shane Kerr. First things first, we had the elections, the Programme Committee elections, was all coordinated by Brian, so Brian will make the announcements of our two new or re‑elected PC members.

BRIAN NISBET: Hello. Once again, I really want to thank everyone who put themselves forward, it's ‑‑ again, this community runs on volunteering and without volunteers we wouldn't have these meetings; it would be the lovely, lovely NCC staff sitting in a room going, this is very nice. So thank you all very much. So we had four nominees, Falk, Leslie, Isabel and Dimitry and after the elections, lessee and Maher Isabel are elected to the PC, so thank you very much to them and thank you to the others for running.

We will be putting all the stats and all the details of the election up on the website sometime next week when we have recovered. So yes, thank you all very much.

BENNO OVEREINDER: Thank you. So, I would like to invite Geoff for this presentation, measuring IPv6 performance.

GEOFF HUSTON: Me again. My apologies, but you have to understand a little bit about the way APNIC works. I managed to come to these meetings if I submit something to the PC, right? So I submitted this talk quite some while ago, to get APNIC to go, yes, you can come to this meeting, success. Earlier, in fact about a week ago, there was that Dyn attack and I thought, that is really interesting, so I submitted a lightning talk about that as well. And I find myself speaking either side of the break this morning, so if it's too much of me I apologise. It was a surprise to me as much as to you. But yes, my name is Geoff Huston and I am with APNIC, this is talking about v6 performance. I was at a talk at actually the APNIC meeting, where a gentleman from Linked‑In was going v6 is so much better and faster, and I keep on hearing this kind of, oh v6 is faster, v6 is slower. And it's just words, and I thought, maybe if we try and make the questions, just that tiny bit more precise, we might have actually a shot at answering it. And actually uncovering from the words what this really means. So, I want to actually look at two, I think quite difficult questions: How reliable is v6 as compared to V 14, and how fast is it, as compared to v4.

So, what I'm really looking at in terms of reliability, is actually the reliability in the workhorse of the web, TCP, you know the transmission control protocol. Because it's actually easy to see it. And in terms of fast, I am looking at this connection, this sort of relative question, I don't care if you ‑‑ completes in 20 milliseconds or 300 milliseconds but if I do something with you in v4 and v6, who is quicker? So can I set up the two protocols side by side, just me and you, and measure who comes out faster? So that is the two things I want to look at.

I don't know if you have seen any of my talks before, but this on‑line ad is kind of the ad that keeps on giving. It is remarkable how much data sits inside a really simple thing, and the really simple thing is that sitting behind the ad is a set of URLs and your browser fetches them. That is it. It's all it does. But on the other side, on the server, is a lot more work. So if you live in Algeria, I reckon according to Google, you would have seen this ad a few times because for some reason this ad goes into Algeria like crazy. The richer your country, the less likely you will see this ad because I'm cheap. And I could keep the price of the bid price really low. And we have to do some post‑processing to get over it. But that is what the ad looks like, it's just our logo and thanks for helping us measure v6 in the Internet or something. Don't click on it. Do not click, okay? It doesn't help, apart making the ad cost, because the impression is a lot cheaper than the click. Geoff said not to click it, is true. Don't click it.

As long as you don't click it, right, don't click it, what actually happens in your browser is that that list of domain names will get downloaded from my server to your browser as a set of things to do. Now, those names are kind of special; they have random components, so it will defeat all forms of caching. It has the time of day embedded just there, so that if ever I see a query from 3 years ago, and I do, I'll know that something is really wrong in your networks, right? So there is random bits, there is bits of the time, there is enough there that I can track what is going on, and I do. I do it via full‑blown TCP packet capture, so TCP dump. So I actually listen to everything that comes back to me, including frags, ICMP, the lot. So I get a very complete sort of picture of what you are doing, by looking at what I see on my systems. I have a number of these servers; currently there is one sitting in Frankfurt just over there in /HA*EDZ and one in line node in Dallas, one in Singapore, and with the help of the good folk in Brazil, we are setting up one in Brazil very soon.

Google ads are amazing. I think it's the one thing that actually sees all the Internet all the time, and I get 7 million impressions a day and they are all different IP addresses. These on‑line ads have really been a modern miracle and that is the tracking point from over a year ago, it's relatively consistent how much these ads just see absolutely everywhere. So we look very, very closely at the way it works and we get a relatively large sample of the Internet, I am confident about the numbers I see. I think they are real. So let's look at the first question, you know, how reliable is this TCP thing? Do all connections actually succeed? Now, how do I know if it fails? I am looking for a very particular kind of failure that I can see, that is not all kinds of failure, but it's the failure I can see.

You send me a SYN because you have done the DNS dance, you know you have got to talk to me and it's got to be http or https, it doesn't matter at this point, it's unencrypted. You send me a SYN and I will send you a SYN ACK you need to send me an ACK, what if you never send it, my path to you is busted or your path to me is busted, but that is not true because you sent me a SYN. So, it's unlikely that you can send me a SYN and not an ACK. So it's more likely my path to you is busted, so what I see is just the SYN and nothing else from you.

How much of that do I see in IPv4? Consistently, across the world, the average is around one in 500, .22%. I reckon that is really high. Think about this in a five‑nines world that is one nine. And that is what we are managing. 98 ‑‑ 97.8%. Not too good is it, two nines. That failure rate is way too high and it's weirdly consistent as a global average and, okay but the real question is, is v6 more or less reliable than v4? So if that is the v4 rate of connection failure what is the v6 rate?

Why does v4 fail? I think it's asymmetric failure; I think it's the routing system and the whole issue of default. But that is another talk for another day. Oh, I did that back in Copenhagen. So I think that is sort of what is going on in v4. But v6, well the first thing is v6 is shit. The major reason why it's such crap, is because a few of you are still running 6to4; stop it, you will go blind. It's shit. The top blue line is just how lousy 6to4 really is. If you are running Windows take it out the back and smash it, whatever you need to stop 6to4 running, it is complete nonsense. Its failure rate is abysmal. The bottom two lines, the green is the average between 6to4 in Unicast, the red line is Unicast, that looks pretty good, doesn't it? No, it's not. It's not .22%, it's 1.5%. So, that is a biggish number. Of a global average, and it's consistent, a bit like v4 only seven times higher. What is going on here? So, 1.5 is unacceptable, what is going on? Whenever you have got v6, always blame the CPE until you know something else. You know, it's just a general rule that kind of works. So, I will blame the CPE and a combination of a bit of strange firewall filters, possibly the rest but CPE is more likely. So now let's actually look at this a bit more. You all might be either a customer of a network or run a network, and the real question is not what is the global average; the real question is, am I better or worse than my competitors? Right. Let's see. This is the failure rate across the world, and if you can't tell, Kenya is pretty bad, Paraguay is pretty bad, Libya is bad, China is not very good and Sri Lanka is where we were with APNIC is terrible as well, even Mexico is pretty bad. Here is the list by country. And top of the list is Sri Lanka, the I talked to the gentleman from Sri Lanka and he blamed it on CPE roll‑out, he is about halfway through which is about right, failure rate is about half. I suppose he will know when he is finished. True. Israel is also abysmal, Mexico, 18%, so some of these countries you might see a very decent roll out rate in v6 but it's not working for people. They might have got it somewhere but it just doesn't seem to be actually working. And by the way, New Zealand does 5% failure. Australia doesn't. We don't care about anyone else.

Your network, Telecolumbus in Germany, someone here? You shouldn't be, you should be back there fixing it. Because that failure rate of 70% means you are wasting your time even thinking about v6, go fix it dialogue in Sri Lanka, that is the chaps who are working hard. Cable com, UK and Netherlands, you have got a problem. And if you see your name on this list, it's not there by accident, and the sample number on the far right is high, that is really, really true, there is a huge problem going in your network, go fix it. Because it's pretty obvious there is a problem and it's near you.

Now, you might want to argue with this measurement, oh we are much better than that and you could be right. I am looking at a single shot TCP connection attempt, just one shot. So it's more likely there is some kind of CPE problem than a network problem; it's more likely it's out near the edge, but nevertheless, it does reflect the user experience in your network. So I suspect this isn't network anomalies, it's edge but something that you and your customers have a problem with so it's worth fixing.

So that is reliability, is v6 as reliable as v4? In some parts of the world incredibly no and more maybe in other parts but it does differ a lot. How fast? Is v6 the magic silver bullet? I love these TCP SYNs, they give you so much information. That is the actual three‑way packet exchanges, SYN, SYN ACK and ACK, from the service point of view, the reception of the SYN and the reception of the ACK is one round trip time. Now, I know that with a reasonable amount of confidence, because inside your stack, that connection is almost at kernel level, it's very hard to interrupt it with other activities, it's not a browser theme, it's actually a TCP driver thing. So that timing coming in off the server is relatively consistent. So, I think I'm getting a shot or a visibility at what you would see if do you a ping long enough and found if you were the smallest RTT. The TCP will give you a one shot RTT measurement which is not bad. So, that's why I use SYN. Isn't it wonderful?

I am moving through pretty quickly. So what I do is exactly the same for failure but this time, this time, if it succeeds, which it does, that is what I am looking at, I am going to succeed in at least two URLs, because almost everyone has v4 and some of you have v6. So of the ones that have v6, I have two measurements, same you, same me, different protocols. Right? Now, if we have all doing our job right and we are professionals, should the path in the network be the same? It's the same end points. So intuitively would you kind of go, well yeah, it doesn't matter what kind of car I drive from here to Seville, it's the same roads? It should be the same path, and the RTTs should be the same, shouldn't they? So let's just subtract one from the other because I don't care how long it takes, I just want to know the difference.

Now, sometimes the world is far ‑‑ weird her that you could imagine. I am sitting in my office and bench testing this, I do a test in v4, I actually look at the path, my packet is to the serve I am using, go to Singapore. And obviously the closest geographic way to get there is via the west coast of America. We are professionals, we do this for a living, the rest of you are just pure amateurs. So, what about v6? For excitement sake it is asymmetric. There is a cable from Perth to Singapore and for v6 I use it to get there but to get back no, no, no, you are going along way, I am so privileged. But you see what happens is that the RTTs are incredibly different, and in this particular case, my measurement shows that v6 is a 112 milliseconds faster. Now, before I go into the figures, just remember this: To get a difference of 112 milliseconds, one side had to bounce across the Pacific ocean and back. That is a really long distance. Right? Just remember that. We will get back to it.

So here is the results of averaging the world, that is a scale of 1:100 milliseconds, that is global average, on average in the ‑‑ in terms of this, v6 is around 20 to 40 milliseconds slower. Wow. That is a lot slower than people say. So on average, we are not doing that good a job.

What is the variants? Really high. That is the mean standard deviation of that number, there is a massive amounts of variant, sample numbers per day, on this particular case I have got 250,000 samples per day v6 and v4 pairs where I know the end point is the same. So it's a relatively large connection continuously monitored. But this is the important one. This is actually spreading all those samples across, understanding where they lie. So in actual case, in this case, around half of the samples, it's the same, a little over half, or under half, are out to the right, where v6 is slower. And there is this other sort of strange peak at around 20 milliseconds and another strange at 125 milliseconds. But you know the world is the world and no one cares, right? But you care about your country. If you are living in oh, Venezuela, China, bits of Indonesia, even Mexico, it's not as good as it could be. The grey bits are actually what matters because the grey is kind of they are the same speed. So, let's have a look by network. Like I said, if the difference is more than 100 milliseconds you are doing something so bizarre I can't even imagine it, so whatever Quantil Inc. In the US is doing with v6 they deserve a medal, but it's pretty weird, even in China, 316 millisecond time difference, someone is sending those packets to the moon and back. I see no other explanation.

So, in some cases you see this is astonishing variance makes absolutely no sense, but the sample numbers are huge, that one there, two‑and‑a‑half thousand sample coming from SK telecom, across the Pacific and back down to Australia and back, and you may have spent 200 milliseconds. I don't know what they are doing with v6 but they are having a lot of fun.

What about closer to home? Now, the green is good, red is bad; Red is kind of bad, red is bad, red is bad, green is goodish. Let's have a look at some of these networks. I don't believe that. My server is in Frankfurt, that is in Germany. So how can a German provider make the v6 packet 150 milliseconds different from v4 packet? Where could they possibly be sending it to get to Frankfurt? There are others there, if you see yourself you should be worried. That is possibly more like what I'd see, around 11 milliseconds, there is path divergence and it's significant because you are only going to Frankfurt and that is only 10 milliseconds away, to have 11 millisecond difference means you are bouncing off something a far way away to get there, shouldn't be doing that. Should be doing better job. Could do a better job, choosing not to. Whoops. What about ones where v6 is faster? This is more like the numbers I would see where you have tried hard to make v6 to work faster with faster path choices, so this is a relatively small provider, Primacom in Germany, Orange in Poland, much larger provider and Tele2 in Sweden, decent job, well done. So those are the ones are managing v6 faster than v4 and doing it in way where the numbers aren't out landish. So this is good. Let's drill in a little further, because we can, we are in Spain. The first thing to notice, this is the top ten providers in Spain. The biggest providers. The ones that dominate the market. This is the amount of v6 we see. Hang on a second, I have spotted it, that one there. .01%. Well done. There is another one, Comunitel .01%. There is not a lot here at this point in time. Let's find the ones who are, okay, this is them. This is the folk, 38% from Art Internet, 32 from Iguana, those are low, relatively small networks with relatively small eyeball counts. But they are doing it. What about their performance, what can we see with those? For Spain, it's about the same, so there is nothing weirdly anomalous in what is happening in Spain, there is the distribution, for a fair few of the samples they are exactly equal and v6 is a little bit slower trailing off to the right. Fine. Germany, more fun. More of them, a lot for folk running v6. Up around, what, 30% of the country is now doing it. Decent set of numbers. Deutsche Telekom, v6 is faster than v4 by seven milliseconds, Cabel Deutschland, five, that looks well cool. Failure rate, 1.7%, 1.1%, get it together. Could do better. Could do a lot better. Liberty, 3%, and the next one, Cabel BW 3%. So performance, great, reliability not so great.

There is the difference in Germany, and this is strange peak where v6 is faster than 4 by about 15 milliseconds, for a bunch of folk, there is almost as if Deutsche Telekom has this routing anominally where to get to prank further in the middle of Germany v6 packets go more directly than v4. It's a German thing. And for the rest of them, it's about 0, there is very little where v6 is slower than v4 which is kind of good. So what can we say about the results of all this? Is v6 as good or as fast as v4? Well, yeah, sort of. In the law of averages, it's not that bad, 75% of the cases you within 10 milliseconds users wouldn't notice. But there is no such thing as the average user, for specific cases it can be a lot, lot worse or a lot better. How about robustness, exactly the same. 1.a 5% failure rate varies as much as 45% down to close to 0. We could do a lot better. Because if we are serious about a v6 network, you really should be aiming for five nines. That means your failure rate should be .0001. Or less. If you are really good at your job, that is what you should be aiming at. It could be a whole lot better than where we are now.

So, if you get a connection running, big gamble, will it be as fast? Most of the time. Will you get a connection running? If you live in Sri Lanka, maybe not. If you live in Mexico, maybe not. If you live in Germany, yeah. It varies depending on where you are. The odds weighted heavily in favour of v4 so. That's it. You can find yourself. You can find your own network, go look it up, see how you rate with comparison to everyone else around you and your competitors and try and make the numbers get better, otherwise I wouldn't bother doing this. If they go up and to the right, in terms of failure, that is wrong, you want failure to go down, go low, five nines. Thank you very much.
(Applause)

BENNO OVEREINDER: Thank you. It's interesting, fascinating, we build a system, just object of study and research, beyond rocket science. Questions?

SHANE KERR: So thank you, Geoff. Very good presentation, as always. So, I am Shane Kerr from the Beijing Internet Institute and I noticed a few observations that you made about traffic in China and I think we need to be careful about those results, because of the way your measurements are selected using Google ad words and basically Google doesn't exist in China basically because of reasons. And also, IPv6 basically doesn't exist in China, either. So, I mean, there are of course ‑‑ you are able to measure it, there are of course many people using it. But as a commercial product, as something that is really possible to get, it's just not possible, so I wanted to make that comment.

GEOFF HUSTON: China is a challenge and there is lot of users, massive DNS blocks and a whole bunch of popular names including Google. Lots of folk do watch YouTube and ads get delivered. I notice in term time I get more ads than holiday time, what a coincidence. I also see a lot of hits inside their academic and research network, so you certainly have a limited degree of visibility. But it's difficult with the Chinese because they take a proprietorial view of their national infrastructure and being an external observer you are never quite sure what you are seeing. If they only had 3 million people in population it wouldn't be statisticically important, they have so many people, whatever is happening in China starts to had a have this massive trend on world average numbers, and yes, you are right, it's a tough one. Right now, what I see is not encouraging. You know, the tunnels, the other efforts folk go to in China to make v6 work are heroic but v4 is faster. I would only hope it gets better. But what can I say?

BENNO OVEREINDER: Thank you. Two questions, over there.

SPEAKER: Thanks, Geoff, for the presentation. I am Carlos from the Portuguese NREN. I couldn't help noticing that you mentioned 6to4, so at the point in time what do you think we ‑‑ everyone should do about 6to4 tunnels and all of those?

GEOFF HUSTON: Blackhole the relay addresses and shut it down. Only run it if you hate your customers. It is incredibly annoying. If you are running something like Explorer it takes 21 second to dig yourself out of the hole that 6to4 dug. No one is that patient. In a billion‑dollar microsecond world, 21 seconds is death. Shut it down and turn it off and do something else. Is that clear?

SPEAKER: Networks institute. Excellent talk, I really enjoyed it. And for packet it's really difficult to travel for 200 milliseconds but it's very easy to see it in a server or router. So do you have any idea of the breakdown with delay, what might be contributing to it?

GEOFF HUSTON: I have heard all these buffer bloke presentations and all this idea that packets spend eons inside buffers, but I do not believe it. Quite frankly once it leaves the customer nobody wants to keep your packets. You might like them, no one else does. They want to get rid of them. So in essence, most of this appears to me to be path issues where taking circuitous paths and part of this is these days it's not just IP on /TPAO* on MPLS and SDN and a little bit more IP with bit more circuit switching and doing multiple layers of abstraction and virtualisation in your networks. This is fine as long as you look at the traffic to the well‑known destinations. But corner cases like me, and I am not hosted in a popular place, I'm kind of the cache miss, and what actually happens, I believe, is that the paths that folk take to get to where my servers are are circuitous but I am not nothing special, I am one of you. Why am I a second class citizen in v6 and are you? And so I suspect the real issue is, we are optimising for some at the cost of others, we are concentrating on path efficiency to do Facebook, Linked‑In, Google, everything else is the slop of MPLS, SDN and everything else, just makes bad decisions.

SPEAKER: What about the server side, like ad network server, do you think ‑‑ might be there as well?

GEOFF HUSTON: Once the ad is given to you, the browser, Google has nothing more to do with us. You and I are having a conversation and a DNS conversation, we are having a web conversation. It's you, the network and me. Eyeball to eyeball, okay? So, no one else has any part of this; the performance is the network between you and me. So, it is a pretty clear view of how the network is doing, what we see as a result.

BENNO OVEREINDER: Thank you.

GEOFF HUSTON: Thank you.
(Applause)

SHANE KERR: Great. So, now we are going to begin our last set of lightning talks for the RIPE meeting and I would like to start it off by introducing Gabriel Lucas, GUIFI.net which is a very interesting effort going on here in Spain, a community based networking.

GABRIEL LUCAS: Hi everyone. Thank you very much to invite me here, I would like to thank Joe and ‑‑ because I am not an expert, I am not in RIPE, but I participate in a network here in Spain called guifi.net that I think it's really powerful as an idea, so I just volunteer in here.

So, what is guifi.net? It is a community network started in Catalonia 2004, build by individual citizens, public administrations and private companies. Right now, it's mainly ‑‑ it's mainly presenting to the Catalonia Valencia, it has been rising quite a lot in the last years since 2004 and those are our figures, the CAPEX it's around €7 million, and the OPEX nearly it's around €3 million. We have around 32,000 nodes and around 18 optical fibre POPs. It's mainly our links are mainly wireless, but we have also fibre links, they started, they have been early deployed in 2009.

At the same time, there was created guifi.net foundation at promote and develop this network as a common so the idea is to have a free, open neutral telecommunication network. We got surprises, one from the cat loan Jan ‑‑ 2007 and the other one last year by the European Union, it was this European broadband award.

So, why it has been successful: This graph, as you can see here, it shows the penetration of Internet divided by regions in cat loan I can't, so for example, the first line you can see how the region of Osona, the place in which guifi.net was created, has higher than ‑‑ it's higher than the mean of European Union Catalonia or Spain. And if you look bit closer over there in the red line, that is the broadband penetration. The blue one is a model or Internet access removal so you can see areas in which people have better broadband connection that access to the mobile. So it shows how guifi.net is getting into those areas and making people access to the Internet cheaper ‑‑ more cheap.

Okay. So I don't have to explain you this slide because all you know the different levels, but our approach is to start building networks in the last mile, so to have everyone connected and when we have enough density of people over there we get into the next level, so in Catalonia there is this public open access network called SOC, so we we get there ‑‑ in which we have the peering with the different ISPs.

So, and this is I think one of the most important parts, and what differentiates GUIFI network from oh, it's based on the commons, I don't know if you know about this woman, someone here knows about who is this woman? She was Nobel Prize for economy, the first one to win an economy Nobel Prize, she studied how communities manage common resource such as fishery, woods, water, for example, through long periods of time. So, for example, here in Spain we have a community that has been managing water for five centuries and she study why those were successful at maintaining this resource. So we are trying to analyse why ‑‑ we are trying to adopt this ideas and methodologies in our network. So ‑‑ and also the ‑‑ one of the ideas is to have this network, so anyone can access to the Internet and we have a means to defend this human right.

So, these are the three or four ideas that are in the core of the guifi.net network, it's open, so anyone can access to the network. It's free because ‑‑ it's ‑‑ you can access knowledge of the network and so you can see how the network is done, so you can technically see what is the traffic going through the routers, what the routers are situated, the whole maps of the network. So we have all these tools, so anyone, not an expert or expert can join and start and connecting to other nodes. Then it's free so everyone can join, you can join as an individual but also you can have an ISP or maybe you can be a little council in a town in Spain and you can join and follow a licence similar to the GPL but adopted for networks.

So, if you know the open access network model in which public communities and municipalities build their own networks and they have a platform in which ISPs can start competing for the customers, this is the idea behind guifi.net network, but instead of municipality or public institution managing this, we have this idea of the common group that takes care of handling these two parts, the physical infrastructure and the active infrastructure. Then, over this area, you will have the ISPs and also community services, that means that ISPs can sell Internet access to whatever people they want, and then there could be also people having other type of services on top of this network, that those can be accessed without paying, for example.

So, in this slide, you saw this part and that is important part, the CPR, but here those are the tools that explain how the common resource pools has to be managed. So the foundation takes care of handling, for example, conflicts in between ISPs, when an ISP, it's put in a lot of money and thinking that others are taking advantage of that, so that means that this should be a conflict resolution method, also there is this monitoring I was telling you, for example, ISPs have to put what the expenditure in the network is so the foundation at the end of the month can know how much money the ISP has been putting or extracting from the network. So, yeah, there is a lot of collaboration agreements, so it's multi layer approach so it's not just technical, it's also political, somehow, very social. Those are the relationships that occur, so, for example, you would have customers that connect through providers, but also volunteers can give support to customers or other people, so, for example, if you don't want to pay maybe there will be a link, public administrations can get into the commons. So this is ‑‑ I just wanted to give you a broad idea, ten minutes is not enough to explain all this, but here you have a few links, this, for example, is quite interesting, 21 pages of explanation of how guifi.net and also more technically, for example, and for here you have to see this video, this was a video done for the European broadband award. Big thanks, sorry for my English, first presentation with so many people, hope you enjoy it.
(Applause)

SHANE KERR: Thank you very much, I found it very interesting. I think your work is very impressive. We have time maybe for one question, if anyone has a question? Going once, going twice. Okay, well, then, thank you very much.
(Applause)

So, for our next lightning talk I would like to invite Evgeny Uskov and he's going to be talking to us about BGP for humans, this is a tool that they built to provide better tools to ‑‑ well, better way to see the BGP table, and he will talk about that, thank you.

EVGENY USKOV: Hello everyone. So, global looking glass or BGP graph for humans. Looking at the title of this presentation, you may wonder why do we need one more such tool? There exists a number of them and I think even on this RIPE meeting there were presented some new ones.

To answer this question, let's consider the following problem: But first, so, suppose that you are a network specialist and you want to know how your prefixes are distributed across the Internet. Or, in other words, how your prefixes are seen by different by autonomous systems in the Internet. So, how can we answer this questions, and what tools can we use? First, we can find a number of Looking Glasses, query them all and analyse their output. The second option is probably to use some existing BGP visualisation tools.

Okay. Let's see if it's that simple. First consider using Looking Glasses, and here there arise some problems. First of all, of course, Looking Glasses are not available for all autonomous systems in the Internet, only for a limited number of them. But suppose that we are lucky enough and we have found a number of Looking Glasses that we need, next what is a looking glass? In most cases, it's just an interface to a router or a set of routers and so a single query to a Looking Glass, usually only shows how our prefix is seen by these particular router. But if autonomous system is big, it usually contains many routers and usually many of them are available in Looking Glass, and different routers may have different AS paths. So if we want to get a complete picture, then we need probably query them all or at least each router in the region. If it's still simple, then let's suppose that we need to query multiple autonomous systems. Many Looking Glasses have non‑standard output, some of them require captures, and probably the last shot is when we want to test some policy change, how it affects the Internet and in this case we need to repeat the situation several times. So Looking Glasses are sometimes a little bit time‑consuming.

Okay, let's try some common BGP visualisation tools. One of them is BGPlay, let's play it and see how our AS is visible in the Internet. Well, looks a little bit complicated, isn't it? It's a little bit more information than I probably needed. Well, maybe we should try Hurricane Electric BGP toolkit and see if it gives us a simpler graph. Well, it seems this is not the case. You know, when I need to analyse such graphs, I feel myself like a solving one of those well‑known puzzles for kids, like this one; well, it usually times sometime for lion not to get hungry. Existing BGP visualisation tools are usually not very convenient or even suitable for solving our problem. First, the problems is that most of them are not updated in realtime and so we cannot use them to see how, to test our policy, how policy changes affect the Internet. And as we have seen ‑‑ have just seen, they usually ‑‑ they may produce quite complicated output, and it will maybe take some time to analyse it.

So, what do we suggest? Our visualisation approach is based on the following simple idea: That each AS path can be written as a sequence of the following form: AS number and AS path length, just like in the following example. Evidently, this forms are equivalent and can be transformed one into another. So let's now use this approach to show multiple paths on the graph. We will put each autonomous system on a separate line and put each node to only certain columns, corresponding to the AS path length at this point. For example ‑‑ for instance, in this example, the first autonomous systems goes to the zero column, 2828 goes to the column number 2 and NTT to the column number 3. And one more point is we don't need to show all the paths that we have, but only those paths that are interesting for the user, that is the paths that user show ‑‑ selects by himself.

We implemented this approach as a realtime graph tool, and as follows from its name, it's being updated in realtime, so all the changes are shown rather quickly. A few words about its architecture. We collect data from a number of BGP sessions, analyse them and transform into a special model representation which contains information about all prefixes and paths we see. Then, we use this model representation to generate snapshots of all paths and Delta files and then the letter data is used to build and display the graph and to update it in realtime.

So, let's see how this works in practice. This is our graph ‑‑ this is the graph which shows how our prefixes reach several tier one operators. You can see here the points that I mentioned. I mean, each line corresponds to a single autonomous system, and each AS is put to a certain column, which number is equal to the AS path length at the current point.

The next moment is that we don't show all the paths; only the paths that are chosen by user and if ‑‑ by default if the user doesn't show anything we show parts for all tier 1 operators.

However, if the graph still looks somewhat complicated and maybe reminds you about the puzzle with hungry lion, then we can highlight the path of interest and the pop‑up window shows all the paths for this particular target.

So, finally, to summarise, the features of the tool include, first of all, its realtime updates, all changes are updated in realtime, then next user can specify the set of targets of targeted autonomous systems, and these two features allow to use the tool as some kind of global Looking Glass, and we try to make it as simple and readable ‑‑ as simple and readable as possible.

So, the tool is available at the following address, radar.qrator.net, free available for everyone and even registration is not needed. So if you have any feedback, please share it with us. There is a contact link on the website. And if you want to improve the quality of the data and especially the quality of your data, the data of your autonomous system, then you may establish a BGP session with us. That's it. Thank you for your attention. I would like to answer any questions.
(Applause)

SHANE KERR: Great. Are there questions? I see one there.

SPEAKER: Massimo, RIPE NCC. First of all, thank you for this great presentation, I'm the develop of BGPlay JS and just as a comment: In the ‑‑ you have your ‑‑ you are right sometimes can be really complex but we introduce some options to prune the graph, so at the bottom there is option you can prune it. We introduce the realtime version around one year ago also, and now is working. We have a prototype in Greece, but it's working also with other dataset, and it's OpenSource also. And yeah. And that is all, just as a comment for ‑‑ and the tools, they do similar stuff, and I would like also to interact with you later and I think it's a really great presentation, great talk. Thank you. That is all.

SHANE KERR: Great. Thank you.
(Applause)

For our last lightning talk we are going to have something a little bit different, we are going to ask our stenographers, who we have enjoyed for many meetings now and they are going to come and tell us a bit about what they do and how they do it, so let's see.
(Applause)

MARY McKEON: I can see everybody, I was hoping it would be too dark. Now I have to pretend you are all naked. I will go back to being dark. The magic of stenography we have been asked many times to give a talk on this and I can't believe what I am going to say next, in the company I am in, at the meeting I am in, at, but I don't want to get too technical about this.

Right. This is what we call a stenography machine. As you see it's a blank keyboard, on the right we know what the keys are, there are 18 letters, 24 keys. Combinations of keys, make different sounds, it's basically shorthand on a machine. The actual definition of shorthand is: The action or process of writing in shorthand and transcribing the shorthand into English. Nowadays we do it by computer. A long time ago it was on paper and you had to dictate it. So it's the amount of that we press the keys up and down as opposed to on a QWERTY keyboard you type one letter at a time and spell things out. So it's all done by phonetics. We can make any stroke on the key, keyboard represent any word, so, for example, Aoife down there will right one stroke, and up will come supercalifragilisticexpialidocious.
(Applause)

So, try write that on a QWERTY keyboard or on phonetics so you have to have a short form. Sometimes you see it coming up on the screens what we call an untranslate, so our software looks for the shorthand outline and translates it into a corresponding English word, if it doesn't find anything up will come an untranslate, I am sure you have seen loads of them, we are so good there is only one or two. So what the scopist does at the computer is put that outline into what we call a dictionary, which we build up over many years so the next time we stroke that key stroke it translates into English.

Your average person speaks about 200 words a minute, it used to be about 160 to 180 but now gone faster and it's getting faster all the time. At the RIPE meetings, it can be so fast just I think actually smoke comes off the machine and our fingers. I actually think it's because your brains work so fast the words can't come out of your mouth fast enough. That is another thing we don't like when hear somebody stands up and goes I will be as quick as I can.

This is the actual, what we see, that is shorthand, for all you technical people that write code it's a form of code and on the right‑hand side is the English version of it. What we do at a RIPE meeting, that is the technical bit done.

Somebody as, you know, what we call the writer sit down at the machine in the hall and listen to the sound, it goes up to the main computer where we have somebody else sitting so puts the words and stuff that we have not heard before into a dictionary that we have built up over time.

Our first full RIPE meeting was in Berlin, so this is our 17th one. I don't now if any of you were at the RIPE ones I am sure you were, but we were traumatised for about a week afterwards. Not alone do we deal with the acronyms of bip and bop and Tap and Tipp and top and IETF and, iBGP and BGP and, we had accents to deal with as well so we were definitely traumatised. We take things down phonetically, I remember people laughing in the room, Olaf coke man came up, and because we took it down phonetically, this is what came up. Oh laugh coke man. So, that kind of shows what we do, we take things down phonetically. So Anna, or whoever is up in the computer puts it into the software, so the next time we strike it at a RIPE meeting, it comes up correctly. Olaf Kolkman. Another one was Peter Cock. We had giggles in the hall that day as well. Now it's in correctly as Peter Koch.

So that is basically how it works. The dictionary has gotten larger over time and we have much more technical stuff in it and we will a bit more familiar with the jargon. We still haven't a clue what in heaven's name you are talking about. We go home and people ask us all about the Internet stuff and we go, don't know, goes from their mouth into our hands and nothing in between.

And it takes an awful lot of concentration, as I am sure you can imagine, something like this or courtroom or arbitrations, all these things and it's extremely tiring because it's very compressed into a short space of time. So really, on a Monday morning we love coming to RIPE meetings and are very happy and this is what we look like. And then comes Friday afternoon and we have worked very hard all week trying to decipher all the accents.

And that is about it. That is all I can say, if you have any questions or anything I would be delighted to not answer them.
(Applause)

Does that mean everyone has a question or...

SHANE KERR: Everyone would be unsurprised if I said anything other than thank you. And I see we have some questions and normally I would ask you to be quick but I am ‑‑ I feel a bit nervous about that. But I guess I will start in the back.

SPEAKER: We have seen ‑‑ we have seen Geoff Huston on stage this time quite a number of times, and so I think we should teach the dictionary, what a metalinguistical variable foo is.

MARY McKEON: That will be in the dictionary the next time.

SPEAKER: It's FOO.

LESLIE CARR: Clover Health. Do you use the name dictionary for our meetings as you do for courts?

MARY McKEON: No, we'd have what is called a main dictionary and then for each job we build up a separate dictionary because for example, byte, we wouldn't be using byte in a normal court case or, other things I can't even tell you that would have the same outline for, so no, we don't, we use separate job dictionaries.

SHANE KERR: Three more questions and then we are going when have when be done here. Cone I will be very quick.

Paul from APNIC. I was a learned Internet governance Vince Serf referred to the jurisdictional aspects of the Internet, but aspect came up as AS SPO K E, and I was just wondering about your dictionaries, how big they are and what you use them for.

MARY McKEON: We have a lot of vocabulary and you can put one stroke can be many different words, stroke of a key that is.

SPEAKER: Sasha, do you ever make corrections or is that for the second stage or like do you roll back and have a back space button?

MARY McKEON: What we do here and things like this is realtime, going through really and you sometimes see the corrections coming up on the screen straight away, we have the capability to correct from our own shorthand stenography machine as well, but when we do a court case that is not requiring realtime or anything we do the transcript, we do take the note down and then go off and do the transcript later on and edit it that way. Does that answer your question?

SPEAKER: It does, thanks great job.

SPEAKER: And ‑‑

JIM REID: Your team are doing a fantastic job so well done to you all. My question is what is the hardest thing for you ‑‑ the hardest thing for you guys to deal with? People like me have got a very bad accent or the people who speak very, very fast like, let's say, Kurtis.

MARY McKEON: That is a very good question because I have a couple of people say, you can understand Jim Reid, that is amazing.
(Applause)

I am lucky my brother‑in‑law is Scottish, we don't find your accent difficult at all. The Irish speak fast. I think we can deal with speed if it's clear but if somebody is muttering and mumbling and that is the hardest thing to deal with. But if they are clear it's ‑‑ we can deal with the speed to a degree.

JIM REID: I just notice on the transcript there, Aoife has got the Scottish ‑‑

SHANE KERR: Thank you very much, it's been great and we all really appreciate the hard work you have done for so long.

MARY McKEON: Very welcome, thank you.
(Applause)

SHANE KERR: So first of all, I'd like to remind everyone that you can rate the talks and we really appreciate it, it helps us and the speakers go to the RIPE page and login your RIPE NCC access account and you should be able to rate them all easy. And that ends the portion of the agenda which has been brought to you by the RIPE Programme Committee. At this point we /THAPBD over to Hans Petter and the RIPE NCC. (Hand it)

HANS PETTER HOLEN: Maybe I will hand it over to the Ops team then.

MENNO SCHEPERS: Has Mary got the clicker? Probably.

I am going to keep it short, and I am a bit scared now after her presentation. So, I am Menno from the Ops team, I work at RIPE NCC and do the meetings twice a year with the Ops team. We come here in the weekend before before the meeting, we set up the hotel network, this time it's COLT connectivity sponsor, they help us get the uplink which was relatively easy because COLT was already here in the hotel, so it's easy then to ask them for another fibre. Yeah, and they announce our meeting space and that way we could get our, the network that we build up here out in the ‑‑ on the ‑‑ up on the Internet.

This is, yeah, how the hotel wiring looks like, on the left you see the white wiring is from the hotel and our neat wiring connected to it. It's a bit messy, but it's all done very quick and as people noticed during the weekend, there might have been still some issues with the network before the RIPE.meeting starts because a lot of work needs to be done to get the network stable, stable enough to run a RIPE meeting.

Here is a slide that shows the DHCP leases during the week or the last four days. Yeah, put some numbers there from previous RIPE meetings to show, yeah, it's not a record this meeting, but it really depends on where you are going, per city, per location, how many people attend and how many people actually use their laptops during the day or maybe go out and see the city a bit.

Yeah, and this is the traffic during the week, also not spectacular. It looks very (spectacular), maybe one thing that doesn't look good IPv4 traffic is still a bit higher than the v6 traffic. Here we see the wi‑fi usage per SSID, we had four SSIDs, we had RIPE MTG, 2.4 gigahertz, that is getting less, the usage from that one, so mostly five gigahertz usage. And also the NAT 64 network has been used a lot more this time I think than previous times, it's getting more, and probably it's because people who start using it maybe actually can make use of the network. Many apps now work, and still lot of work to be done there.

This is the wi‑fi spectrum we have, there is five gigahertz, 16 channels and 2.4 gig three channels. This is also the reason why 2.4 is giving a bit more trouble to get stable wi‑fi connection on because there is just too much chatter in there in the three channels we can use, whilst five gigahertz with 16 channels it's easier to spread the load.

Gee location geolocation, I was wondering if people noticed the issues, as in if you were in Copenhagen or in Amsterdam when you opened your laptop or your phone, because, you know, it's ‑‑ it's always difficult to update all the geolocations databases because it's not always clear what people are using, for example what is Netflix using and what is your television provider using, what is ‑‑ all these companies, they use some way of geolocation but it's not always clear, and if you e‑mail them they don't always reply, and those that do provide ways of up dating, do not always apply them. So, you think that you updated them with the new details but then it doesn't work and you reply to them, or e‑mail them and you don't get replies so it's a challenge. But yeah, if during this meeting you saw something that wasn't in Madrid, please let us know at Ops MTG [at] ripe [dot] net and hopefully also with some hints maybe on how to fix that.

This is the ‑‑ just some pictures showing what we have been doing as well, and that is, we set up the webcast for the remote presenting and we got the presentation kit, so all the presentations load nicely. And also the audio guys who help us out. You see also the camera on the right, it's sometimes challenging to mount cameras and this is done with tie wraps, but it's stable, at least it's been proven to work all week. The stenography, I don't have much to say about it any more after last presentation, but we are always very happy with having them around. This is the Ops team, you see us run around, if there is any technical questions you can always approach us. And yeah, if you have questions, now, please feel free to ask them. Thank you.
(Applause)

HANS PETTER HOLEN: So, thank you very much for, as always, doing an excellent job. I hear a lot of people saying that this is the best conference network, this is the best technical set‑up we ever go to and I even gave away the remark over dinner it's actually better in the office I work in at home.

So we have come to the end of RIPE 73 here in Madrid, I have some closing words. We had 630 attendees checked in so that is really good. Unfortunately it's not a record. So we have the slightly more in RIPE 72 and even a couple more in RIPE 70 so we still have to work on growing if up and to the right is the goal in life. But it's been really good. We see that we have a healthy supply of newcomers, so we are not running out of attendees shortly. 28%. And that is actually in small increase from last time. And as always, we would really like to have feedback from you, there is a survey on the website that you can go and click and we actually draw some prizes so you can win Amazon gift voucher if you answer this feedback form.

Looking at the attendees, we see that the biggest country in the RIPE region is still to the west, which if you know the history of the US you know that there were a lot of Europeans who moved there a couple of 100 years ago, but it looks like Germany and Spain is high on this list, I didn't get this exact list this time but it's really good to see we have participants from larger part of the regions even though the east most part we would really like to see even more participants moving forward.

Type of organisations, a healthy mix there with both commercial associations, education, governments and we also have a good portion of friends from the other RIRs coming to visit us.

Then, as I am growing old, you have initiated a process to replace me. I had a suggestion very late one evening that we should have a process to establish a process to replace me and that that process should take 15 to 20 years, so I take it that there is not really a hurry, but we have started the discussion. There is a URL here where the proposed text that is out there for the role description and for one way of doing this, as well as a mailing list, so click on this and make sure that you subscribe to the mailing list. We will work on it there and then maybe if there is a need, we will have sometime to discuss it also at the next meeting and we will take it from there. It's no rush, I don't want to run away from you; on the other hand, I don't want to leave the community without a way to replace me in the case that is needed.

So, then I would like to thank our hosts, so Joe and Cristobal, do you want to come up here, I hear we would like to give you a token of our appreciation. And since I can't see anything, I don't know if anything is happening here so that is great. Thank you very much.
(Applause)

And I take it Cristobal is not here? Okay. Then we move on to also say thank you to our sponsors. As, you know, we wouldn't have all the nice coffee and the parties in the evening and so on without sponsors, so we really appreciate that.
(Applause)

And then this meeting would definitely not happen at least not the Wednesday and Thursday, without our Working Group Chairs, so I would also like to extend my sin sees thanks to the Working Group Chairs or putting together the programme but also to work with the group on the mailing list in between. Some list vs. More activity than others and some discussions are more heated than others so I would like to remind everybody we work towards a common goal here and want to be nice to each other and please give these guys a big applause because they really deserve that for looking after you in between the meetings as well.
(Applause)

And I want to extend a special thanks to some of the Working Group Chairs who are stepping down. I don't know if you are here, we have a small token of appreciation for you. I don't think Anna is ‑‑ Anna, Jim and Job, you are here. I can see you. So Jim has been chairing the DNS Working Group for, I don't know how many years, Jim?

JIM REID: 15.

HANS PETTER HOLEN: So thank you very much for your tireless efforts in keeping the DNS running.
(Applause)

And Anna and Job, they are not here? Okay. So they have been replaced by Shane, you have seen him as part of the PC here, so he needs no further introduction and we know he is working with DNS so the DNS Working Group should be in good hands. Raymond has stepped into the IPv6 Working Group. Achilleas and Julf have taken over the chairing of the Cooperation Working Group. So thanks very much for volunteering for that work.
(Applause)

And the RIPE meeting would certainly not be what it is without the Programme Committee, so I would really like to extend the thanks to the RIPE Programme Committee, and shared by Benno and with Shane at his side, they are sitting here in the front row, but the real thanks now this time goes to the outgoing PC members, who is the Marcus, I don't know if you are here, Marcus? No. Anyway, thank you, and I would like to welcome Maria and Leslie, and Leslie has been re‑elected, so you know her, so welcome to the PC. For those of you who didn't get up in the wrong, Filiz has been re‑elected to the NRO NC, so congratulations, Filiz.
(Applause)

And again, thank you to the stenographers. I am trying to speak slowly and calmly this time, I don't know if I can do it clearly but I can try to improve for the next meeting. Thank you very much for your effort.
(Applause)

And then somebody put something in this slide that I didn't know about.


NIGEL TITLEY: (I don't speak Latin.)
(I don't speak American.) (I give up.)

(Secret Working Group).

HANS PETTER HOLEN: Thank you very much, we have a couple more prizes to hand out. Ondrej, you are the proud winner of the one minute LIR quiz.

SPEAKER: With 11 correct answers we thought we were dealing with a super human but he confessed to have doing the LIR course /PH the academy the night before so that explained things.

HANS PETTER HOLEN: Then we have two more prizes, if you have been to RIPE meetings before, you will know that we usually pick a number of the first registrations, but then there has been some suspicion of foul play or reverse engineering of the publishing algorithms so we discussed a bit on whether we would change to the last registrations /TPWHAU would kind of give the wrong incentive, so we have changed it a bit. The proposal actually came from one of the (but that) one of the winners, or one of the potential winners, so he may regret this now; but anyway, as a twist we will this time hand out a prize to the first registered newcomer who is present at this end so you then you haven't reversed engineered the publishing algorithm yet and me and pronouncing Spanish names. /TKPW*ERD con Val low.
(Applause)

SPEAKER: It's not a Spanish name but a Portuguese name.
(Applause)

HANS PETTER HOLEN: Yeah, not only do I not understand the names I don't know the language either. So then back to the traditional registrations. And I am not sure if we know this guy, he is from a company called Elisa in Finland, Raymond, are you here?

Raymond: Yes.

HANS PETTER HOLEN: And then if Andre cloth /TKPWA is here as well. Those were the first two registrations so registration 2 and 3 because as RIPE Chair I am always number one.
(Applause)

And that brings me to the end of RIPE 73. So, thanks for a very interesting week, and for very good company for the week and see you all in Budapest.
(Applause)

LIVE CAPTIONING BY AOIFE DOWNES RPR

DOYLE COURT REPORTERS LTD, DUBLIN IRELAND.

WWW.DCR.IE