Archives

Plenary session

25 October 2016

At 3 p.m.:

ONDREJ SURY: Please take your seats so we can begin. So hello, welcome to the first afternoon presentation. Well, let me remind you that there is RIPE PC nominations running and you have exactly 15 minutes or so to phenomenon mate yourself if you want to run for programme committee. Come, it's fun. And we will begin with Gregory Mounier with ‑‑ from Europol, who will presenting what troubles we are causing the law enforcement with our Whois data. Please come, Greg.

CHAIR: And a small reminder. You have this first talk, this is your last chance to nominate you to the Programme Committee of RIPE meetings. So, if you are considering this opportunity for you, please do it right now because you have just one talk in ten minutes, 40 minutes actually for nomination. Thank you very much.

GREGORY MOUNIER: Good afternoon everyone. I am working for Europol which is the European law enforcement agency, and in particular I am working for the cyber division of Europol which is the European cybercrime centre. I would like to thank the RIPE Programme Committee for having accepted my proposal to speak to you today. I am going to talk a little bit about the Whois accuracy and the vision and the views we have in the public safety community about it, I think this is really an issue of mature interest for both the public safety community but also the RIPE community. And yeah, I am pleased to be able to share our views in front of you today.


So the objective of the presentation is to give you a little bit of the background of why we are engaging with you as the law enforcement community and also to explain a little bit about the public safety use of Whois. As, you know, the Whois was, the original purpose was really a tool for network operators, however the Internet has evolved and so has the use of the Whois, and nowadays not only network operators are using the Whois but also public safety agencies and when I say public safety agencies I don't only mean criminal investigators, for instance, but also customer protection agencies, individuals, businesses, who are trying to identify abusers of our network and infrastructure and try to find and seek redress. And I will explain a bit about the current Whois accuracy challenging that we have and then I will show you a case example, the idea is really for you guys to put yourself in the shoes of an investigator, working on a case, trying to investigate and manage and you will see the hurdles and problems that we might have, not because of the Whois, because at the end of the day this is a really great instrument to attribute crime but because maybe some shortfalls within some of the policies which leads to an accuracy down the line and downstream for some of the operators.

And then the last point I think that is really what I would like you to gain and to remember from that talk, is that we as the law enforcement authorities and agencies and the public safety community, we are trying to engage with you to try to cooperate and lay the ground for a maturely beneficial policy, our operation is multi‑stakeholder, bottom‑up, we want to engage and have your advice and feedback and try to together improve the accuracy of the Whois.

So first of all, uses of the Whois, as I said before, not only original Internet community, many other actors do use it and I think this is one of the main elements of accountability on line. We are trying to ensure that IP holders, address holders are properly registered so individual consumers and public have power to resolve abusive practices that are impacting safety and security. The other use of Whois of course is the more traditional one, being network of contact for network operators, for troubleshooting and to security and reliability of the network. The Whois is used to help businesses and consumer groups to combatting abuse and fraud and last but not least it's used also to identify and find information and potential bad actors using IP number resources.

We, as the public safety community, we use the Whois in particular as one of many other tools. It's not the silver bullet of course to attribute crime on line but really, this is the most common starting point for any cyber investigations, and in fact, I am saying cyber investigations and I should say any types of investigations because if you are investigating as a Drugs Squad, you are investigating an organised crime group that is moving drugs from one country to another, you will obviously have an IP address popping up and you need to be able to trace that IP address back to some end users at some point.

So, the Whois is a great instrument, really helpful for many, many actors to seek redress and to identify abuse, but there is a problem; the problem we call it is linked to the IP address chain of custody inaccuracy, what do we want to mean by that? We want to say there is a problem with the sub allocation that are not properly documented to the last downstream provider and this is for us a concrete problem that leads to inaccuracy, waste of time and resources. Another problem is linked to the fact that each Internet have different policies and requirements on what information has to be retained regarding sub allocation. And we think that the problem is only expanding, are your T extension every single devices being connected, having IP addresses, we heed to be able to use existing tools such as the Whois to stress it back to end users. IPv6, we understand and we believe that network operators will be allocated massive, big blocks of IPv6 addresses and there will be less reasons to come back to original Internet registries such as RIPE and therefore there will be less incentive to properly record and register sub allocations so we think that will increase the problems. And you have new protocols such as those being developed in the IETF which will allow the resolutions of any phone numbers on the Internet and again, this will increase the problem, we think. So we are really trying to seek an industry solution, trying to find your views and understand your views and trying to come up with the best solution that will fit your constraints and our constraints as well.

The challenges, what does had a mean in concrete to not being able to fine ‑‑ information for us in our investigations? First of all, inability to quickly identify resources that are used in abusive activities. As you know in cyber investigations the evidence are very volatile and can be moved across continents and can be deleted very quickly, they move from one server to another, it's very important to have a tool to be able to identify where those resources that are being abused, are, and locate it very quickly. Inability to serve legal process. I think that is one of the most frustrating elements when you are doing investigations. The fact that you are wasting time and resources to find a proper address for the last downstream provider of an IP resource and you are not able to send that in order to get information on the end user, it's also important to find jurisdiction quickly, if I give you concrete example: Europol is working for the 28th European national law enforcement agencies and we get very often referrals of, for instance, IP addresses that are being used to download or access child sexual abuse material, on line and for us it's very important to be able to determine in what country or jurisdiction those IP addresses are being operated so we can send the information to the national law enforcement authorities and they can take actions. But you don't only use the Whois to identify potential suspects, but you also Tuesday to find victims. Everyone is talking about ransom ware nowadays. We also get referrals from the private sector of IP addresses that have been compromised or infected by ransom ware. For us during investigations we will see server and then we have a number of decryption keys and it's important for us to identify where those victims are so we can also help them. Of course we work with the community and the rest but it's very important to quickly find where those IP are being operated.

Waste time of investigators but not only of investigators, in fact network operators. I know that you are very busy and you are running businesses, and you don't have to deal with law enforcement requests for information if you are not the one responsible for the IP resource. And so, very often, when we do investigations we will jump from one ISP to another, sending legal requests and taking the pain to go to a judge and trying to send that information and then the operators says sorry these belongs to a block of IP which has been suballocated, I can give you the name but the address will be difficult, you can find it, in the Whois. And then of course, then starts the problem.

So, I will now present briefly a case study. So initially I wanted to ‑‑ this is actually a proper case was shared with me by British colleague, I think it's up to date until last month, somebody said it to me about four weeks ago. The initial idea was to give you the opportunity to do the OpenSource research live together with me, basically. So I wanted to give you the IP address and then would you try to find the address of the last downstream provider. Had second thought about it, and I think it's probably better also from a data protection perspective that I keep that IP hidden and the same so I have changed also the names of the individuals that will appear in the lookup. I kept the names of the companies because one of the companies hasn't done anything wrong anyway, and the other one is known actors to be a little bit involved in shady business, so in the end I kept it.

So, the case study is a major supermarket chain that comes to see the police to fill in victim of crime report. The complainant says that they have been, the IT system has been compromised and the result of that compromise is that they have had major data leakage. So, internal investigations find, can source the network intrusion to an escrow injection attack and that attack has resulted in the filtration of 8 .7 million customer details. It's not a major leakage if you compare to other ones those days but that is for a company like this one, this is really, really big issue.

So, internal investigations and going through the escrow log files, we managed to trace back the attack and the event to one specific IP so that is the IP which is on your screen. So from that point on, for an investigator of course the first thing to do, and as soon as possible, is to find a physical address of the last downstream provider that is holding the resource and that address so that we can serve a legal order to attribute the attack to a named subscriber so we get the information and I think I can pause here for the case study and emphasise that we are not after the end user data, we need anyway to follow legal process and we need to go to a judge to require a legal order or production order or rogatory letter, as you want to. There are many different terms but we need to produce legal document to that company so they can provide us information on the end subscriber. But we need to have the address of the provider.

So the first thing we will do is to go and do a Whois lookup on this IP address. From that page, you can see that already the IP address belongs to a /24 prefix, which has been allocated by lease web Deutschland to inferno. So that is a case of sub allocation. The interesting bit for us is that there is a RIPE person with the name and UK contact, that is something we are going to pursue ‑‑ yes that was sub allocated to Inferno, but in theory if you follow the logic of the pups of the Whois you should be able to find it quickly in the Whois database.

So we queried the RIPE database on John dough, the first name that pops up, and it resolved to the original Whois result so that is fine, we still have the address in the UK, the first tricky thing is we find out that the phone number is actually a contact phone number based in San Diego had the US, that is the first indication there might be something wrong. We research the RIPE records for inferno as provider names and contact and address and then we find yet a second UK address and a different RIPE ‑‑ as well. If we click on the RIPE handles we find yet another name and address. So basically, we are after the registered address of official provider called inferno and we have found already after four steps two different personal objects and two UK addresses and one Serbian address and one US contact phone number. But we continue. Reis he search the inferno dot‑name in the domain Whois, and we find that the domain is protected, probably by a proxy and maybe the register is based in rugs I can't, and the network is CloudFlare hosted so not much luck. Then we basically go to the website of inferno dot‑name and we find that there is that Inferno is a registered company of Three NT which is on the website and there is a new address, I don't know if you can read but it says "10 great Russell street" in London. That might be a new lead. So we do search on 3NT.com in the domain Whois and we find another name, with a new address in the UK. That is down ton house. We continue the research on the website of 3NT solutions and on the contact page there is indeed the same address so might be the right one.

But in order to be completely sure we check again with another OpenSource database, the company housed data.co.uk, a tool randomly used by businesses and investigators in general and we see in fact the down ton house 60 Windsor avenue, there are two companies list there had but none of them is 3NT so it's probably again a a false address for us.

So we go back to the RIPE database and we look for Joe Bloggs, the guy, and we find the same company name so that is consistent. So again, good hub. We then try to click on the handle and we find that Joe Bloggs has a different address, this time in bell ease and a different company, D LR A telecom and the phone number is actually based in Sweden so really super confusing.

Lastly, we don't give up and we go back to the RIPE members list and query the 3NT as selector and we find 10 Great Russell Street address so that might be the corporate address that we are looking for. But before going to a judge, before taking the tube and handing in the legal order we do a quick query on the Google map and we find that Inferno 3NT UK is actually Dropbox so it's probably not the right place you want to send your legal order. Again, we do a quick Google map research on Windsor Avenue, that came up several times and we found down ton house could be a server or maybe a drop address so we are not going to send legal order there. And last but not least, that is one of the last slides, we query another publically available database on company house information for Joe Bloggs and that reveals that Joe Bloggs has many different companies, all hosted either in bell ease and if you continue the search you find he has companies in the British virgin islands, Panama and the rest. That is how the investigator will feel at this time.

Seriously, summary on the inferno case. First of all having done all these OpenSource research we are still not 100 percent sure where and who to serve that legal order to provide the name. The common UK is a suspected drop address, 10 great Russell street. Multiple RIPE members records and handles act as very good for law enforcement but a query of records is questionable, as you can see through the different OpenSource entries. Records how are very hard to link together and lastly, multiple company addresses and counter registration points to three different continents so at this point either you drop the case, depending on the severity of the case, I suppose revert to more elaborated techniques but this is just to show you the purpose of the Whois is to find the contact details of a provider, you can't do in that case because of sub allocation.

The conclusion is we want to work with the RIPE community and network developers to develop a policy. We believe there is mutual interest to act and we have a number of suggestions to make ‑‑ Brian Nisbet, the chair of the Anti‑Abuse Working Group has invited me and given me a small slot on Thursday to address about these type of policies so I am not going to delve into the intricacies of our suggestions but basically the main idea would be would be to require registration of all IP sub allocations to downstream provider so the entire chain are currently reflected in the Whois. It is for to you tell me if this is realistic, it is for to you tell me what is the best way to achieve that and if you have a commercial interest to have these type of things, they are alternative like geoallocation filled, in the RIPE Whois there might be some actors that are interested to develop these, but yeah, I think that is the end. That is also how we feel when we look in the Whois, chasing our tails, wasting time and yeah, thank you very much for your attention and if you have any questions, happy to take them, thank you.
(Applause)

ONDREJ SURY: So, I see ‑‑ yeah. Starting here.

SPEAKER: Hello, Alexander. First of all I would like to thank you that you came here and presenting your presentation. I could not imagine a police officer of Russia presenting like this. But even with my experience with local police officers I have a set of comments and questions. First of all, you told about the story much longer that any in this audience could run through all Whois and something like so argument about wasting time is right I think. If a police officer have experienced problems doing all you explained maybe he should go clean streets and do something like this. But not being police officers. Second comment is, all you stated in RIPE database is actually violating now RIPE policy, I think. And it's violating ‑‑ it's come from real life because if we are watching for all the cybercrime scenes you see most criminals are couched in real life. By real life actions, getting money from machines or deliver parcels or something like, so, all this in especially you told about drop addresses, fake companies and police came from real life and you are ‑‑ and you should be accountable and responsible for registering fake companies, something like it says you police doesn't working well in real life allowing these things which have a little projection in RIPE database. So please be accountable yourself, don't talk with RIPE accountability on this. But for question: You are talking about problems in a ‑‑ you gave us one short case, but you did not give us numbers. How much such cases have failed or got problems by Europol, by police officer of any country members? So maybe this is only case Europol is studying. So, your question is out of scope. Having RIPE database and RIPE data currently working for every one of us for years. The question is: How much such cases and problems Europol have? Could you provide this data?

GREGORY MOUNIER: Thank you very much for your question, that is very relevant. I will take the last one before. Most of the cases we have problem with are related to these type of things, so I could say it's difficult to give you specific numbers but I am sure we could come up with. This is one case but I presented also in Copenhagen in May with a different case, we have got plenty of cases. That is two but we can go ‑‑ what I want to say is the community in general, all the police are very well implemented (policies (we don't have any problems with RIPE members, but with downstream providers, because as you say there is no breach of any policy because there is a gap, I think, and that is what we think and we are looking to solve. As to, this is the police responsibility, that is businesses are registering drop addresses and the rest, I don't really follow you here but I think what we are trying to do is the most simple step, trying to narrow down the cracks basically, of course we can talk about .net and VP S and many other ways of hiding your trace on the Internet but you need to start somewhere, and this is a community which is devising policy on how to work together and establish standards and so our thinking is that, okay let's start from the bottom, baby steps and try to reduce the gap by providing and devising policies that makes it more difficult for the bad actors to abuse the resources, because all the good actors that are that room will not be in that situations. It's the other ones that are using the grey zones in some of the policies that will be abusing. What was your first questions?

SPEAKER: Comments about numbers, if you want to return these policies or something like on Thursday, please find numbers and present them.

ONDREJ SURY: There was a gentleman there.

SPEAKER: . So ‑‑ group on ISP which says present in various countries in Europe and Africa, of course I was supportive of such an approach of approach on such reliability and ‑‑ the Whois database, even if it is not very easy so we are supportive because of marketing on technical aspects of our business. So, your question was, so have you support, can you find support about that so I can say if you want to talk together why not?

GREGORY MOUNIER: Thank you very much. Surand then there was ‑‑ yes. Up there.

SPEAKER: Raymond of Finland. First question actually is where did you get permission to use the RIPE database for this?

GREGORY MOUNIER: Well, as I said, the RIPE database is open and public accessible available ‑‑

SPEAKER: And yes and there are rules for using it.

GREGORY MOUNIER: Do you mean for the presentation or for investigation in general?

SPEAKER: No, for anything

GREGORY MOUNIER: For anything. This is publically available instrument so you can use it as you know any public available information.

SPEAKER: No, you are not allowed to do that. Could you go back one slide, there is some other things I would like to ask. So, every member of the RIPE NCC being a LIR, for example, is obliged to register the information in a database, in a correct way, but what the information is depends on what you get, and if data is being updated and if it's being maintained updated, that is usually a problem because, yes, when a customer becomes a customer you get information but never after that. They can move, you know that stuff.

GREGORY MOUNIER: And we as a public safety community, are telling you our opinions and we think that they might be a bit of an issue, but we are happy to engage and discuss that with you.

ONDREJ SURY: I saw somebody there.

SPEAKER: And drew de la Haye. RIPE NCC. Thank you very much for bringing your thoughts to this audience and showing your challenges, as one of the stakeholders, end users of the Whois information, so that is for sure. Another aspect of the high quality registry is securing the right of use of the resources of members of course, and hence the operational teams within the RIPE NCC have been very active in due diligence and examples of those are the ah assisted registry checks in which we contact our members once every now and then to see whether their data is correct and accurate. Tomorrow, during the Services Working Group I will be presenting about some of these initiatives, and the quality of the registry so everybody that is interested in that I would highly recommend to participate in the Services Working Group.

GREGORY MOUNIER: Thank you very much and if I'm not mistaken, the A RC procedure aflies direct member of RIPE, right? And what we are talking about is the clients of your members and for us that is a bit the white elephant in the room because if all your members are doing their homework and they are and that is perfectly fine and we have no issue with them, it's the issue of the fact that sub allocation is not properly registered. And I don't think that the RIPE procedure applies to those downstream providers.

SPEAKER: That is indeed the case and hence we applaud you being here on stage to bring this forward to the community, and to see what the expectations are. Thank you.

ATHINA FRAGKOULI: Head of legal from RIPE NCC. Just a clarification on a previously mentioned point of the purpose for use of the RIPE database. I would like to direct to you article 3 of database terms and conditions. Article 3: Purpose of the RIPE database, the RIPE database contains information for the following purposes: Fifth bullet‑point: Providing information about the registrar and maintainer of inter number resources, when the resources are suspected of being used for unlawful activities to, to parties who are authorised under the law to issue such information. That was clarification for the purpose of the use of the database. Thank you.
(Applause)

CHAIR: And next part of this session is the panel. This panel will be ‑‑ was prepared by Leslie Carr and smaller ISPs will right now show your money.

LESLIE CARR: Good morning ‑‑ or afternoon. So, today we have a panel focusing on smaller Internet Exchanges, and most importantly, not using the pointer correctly ‑‑ ‑‑ technical difficulties. These always happen. The slides aren't really that important. Most importantly, we are focusing on money, smaller Internet Exchanges often have some more unique challenges because they may not have a large organisation to back them and backing them financially. So, today, I would like to introduce my ‑‑ myself, my day I don't be is a company Clover Health but I am also on the board of directors of SFMIX which is a small nonprofit Internet Exchange in San Francisco. I would like to introduce Ulf Kieber from Swiss IX, Zoran Perovic from SOX and Will Hargrave from LONAP.

LESLIE CARR: Tell me how your exchange was founded because as we all know switches and equipment can be a little expensive.

WILL HARGRAVE: This is kind of interesting question because when it started I was still at school in the late 90s, 20 years old, it was started in the usual way and it's not very expensive, the primary resource is coffee and getting people around the table and someone finds a small switch, and our growth since then has been organic so I guess I wasn't involved in the start and that is really the information I have.

ULF KIEBER: I wasn't involved in starting Swiss either, somebody finding a switch and providing it basically. Orb. So Fox was founded in 2009. It's a private company, I bought switches, and it was made for profit. Unfortunately, it took us roughly five years to reach some profit, but then again, my partner, I professor are very well known figures in Serbian market and we started on top of that, on top of our knowledge, our reputation, and we waited quite a long time to see, to show me the money, to see where the money actually is, so if we can have a first conclusion here, if you are in it for the money don't start Internet Exchange.

LESLIE CARR: And speaking of the money, when did your exchange start to be able to have paid staff and how were you able to make that decision?

ULF KIEBER: If I start, Swiss X doesn't ‑‑ at this point in time Swiss IX don't officially have paid staff. We are just in the point where we changed that. We have 170‑plus members and it's grown to a size where it's not ‑‑ no longer possible to actually handle the ‑‑ all the work with unpaid and voluntary work. Orb orb same with SOX, they have four employees, of course my partner and myself and two more people, an engineer and another assistant. We actually worked, my partner and myself, without any fee for quite some time. Then when it grew to, how to say, couple of hundreds of gigabits of traffic, then we started making some money and we started employing people, renting offices and providing more technical support.

WILL HARGRAVE: So the staffing things is quite interesting, we now have four employees, we have had our first employee maybe four or five years ago and we think about the first person you want to employ and you might think we need a network engineer because this is a technical facility, but the first person we employed was an administrator, there is lots of people who want to run switches but people don't like printing out 200 invoices and putting them in envelopes and sending them places, so we actually ‑‑ our first employee was someone to work part‑time to do our administration. And we have ‑‑ we have been able to grow since then with adding technical and manage yearial staff.

LESLIE CARR: That is a very good point about all of the administration, because that is a surprising amount of not trivial but non‑technical involved in running exchanges.

You know, so ‑‑ so, speaking of that, port fees have been sort of a hot topic recently amongst many communities. How did you decide how much to start charging and have you had any big issues with that?

WILL HARGRAVE: So originally when LONAP started we had this 2000 pounds a year membership fee and you paid to be a member and you could take whatever ports you wanted and obviously that doesn't really scale, it's when you are a small Internet Exchange and you have some gigaports and stuff, well, fine. So, yeah at some point we had to start charging per port and charging more money for ten gigs and putting up prices is almost unheard of this in this industry and kind of shocks people but as long as the pricing is fair people do not object too much.

LESLIE CARR: Did you have any members leave when you had to increase prices?

WILL HARGRAVE: We had someone downgrade which is quite bizarre, but I guess if you have a 10 gig port and it has 200 MEGS on traffic on it could you downgrade if you are feeling cheap. That was quite a long time ago now. There is some natural turnover as well in acquisitions and people leaving so it's ‑‑ it's difficult to attribute. We find the cost is not a major factor; once you are on the exchange people tend to stay.

ULF KIEBER: It's somewhat like in SwissIX, at some point you have to realise that hardware has a lifecycle and needs to be replaced after sometime and it doesn't survive forever so you have to have a sustainable business model so you can ‑‑ at some point you can change your hardware and buy new hardware and if you don't charge for the ports then that doesn't work.

ZORAN PEROVIC: In SOX, since it's a private company, we had everything business plan, we had price list, first we installed switches and then monitoring system and sFlow and then we put price list on our website, then we asked customer to connect. But nobody would connect. Then we were giving like 6 months for free and a year for free. So, it's a little bit chicken and egg situation. You cannot have more customers until you have big CDNs and they would not come if you don't have operators. Operators do not want to peer with their competitors in the market, etc., etc., but eventually, and I will probably say that number of times today, communication, collaboration, cooperation is what actually makes a good IXP. So we spent really, really a lot of time explaining, talking, persuading and eventually price is not the issue. Somebody complained that we are too expensive but then again we don't have the scale of links that AMS‑IX, we don't have low cost like university based IXPs, we are a private company and bay for everything, for each and every co‑location and switch, we have to upgrade for dark fibre, we even put 5 POPS in Belgrade in order to be closer to our customers and all of that costs money. Eight years later I can say we only needed two POPs but at the beginning we didn't know that so we had to invest invest invest, and again, we are not in this for the money.

ULF KIEBER: The first issue commune based association has is probably not the business plan, if you have a community based association and do an exchange point you don't start with the business plan, you start with the switch, and start with the infrastructure and the text nicks. Orb orb of course that is the benchmark in region or Europe or worldwide so our customers always compare our prices with prices of other IXPs.

LESLIE CARR: Well, I really want to make sure this is a conversation with the entire community. So, could we get the lights up and I would love to have any of your questions. How about we have the moderators, though, do the ‑‑

SPEAKER: The question to all is ISP holders is there are nature of traffic is going to exchange on Internet Exchange is Internet traffic so we have BGP and so on, but there is a big demand of exchange in VLANs, like if you want to connect to upstream ISP or private very expensive channels like direct fibres. Internet Exchange provide this kind of service? Thank you.

LESLIE CARR: Well ‑‑

WILL HARGRAVE: Actually we do, we allow free VLANs between participants at LONAP. It's not a very widely used service but we do permit it and provide it. There is an important I think that we are not for profit owned by our members and we don't want to compete with our members who run Ethernet carrier services. It may be difficult in your market to get Ethernet market but in London it's simple and could I name five or ten different people that would do you reasonably priced sets of services so there is an element we don't want to provide really generic cheap Ethernet carrier services because it could compete with our members and that is not fair.

ULF KIEBER: We don't want to compete with our own members and sponsors but in the end we do to some extent provide P and Is between members if they really want that and ask for that.

SPEAKER: The question is why not develop a service, why not so common service, was the main problem, thank you.

ULF KIEBER: It's based on member request. Orb orb I can tell you our experience, we provided private VLANs from day zero, so nobody complained that it is competing but nobody used it to connect to upstream provider. Then only recently we saw the boom in request for private VLAN mainly due TV channel distribution or maybe some voice‑over IP interconnection on ‑‑ so out of 200 gigs of traffic, maybe a couple of hundreds of megabits are actually in private VLANs and usually the for TV channels.

LESLIE CARR: And I can't speak for everyone but in our case, for SFMIX, several of our locations are donated and we can't compete with the donor of our space and power because they are ‑‑ they are a data centre provider who also sells cross connects and I think there are several community IXs in that situation as well as the no member competition.

CHAIR: Here.

SPEAKER: Thank you very much. But ‑‑

CHAIR: Name and affiliation.

SPEAKER: Open net. And all your colleagues are mostly from Europe so we have experience only from one continent. But could you answer your questions yourself about American experience?

LESLIE CARR: Oh, sure. Is Matt Peterson here since he is the founder of SFMIX, he would be able to answer this better. SFMIX was started about ten years ago. There was a data centre which had no Internet Exchanges whatsoever. So, basically if could you ‑‑ he asked if he could put in a switch and there you go. We have no paid staff. It is something that many days I wished that we did, especially since we all have day jobs as well to pay the rent, San Francisco is a little pricey, you may have heard. And we just started charging now basically because Matt and his co‑founder had spent a significant amount of their own personal money to run exchange for ten years first there were donated switches but then switches died, they had to be replaced, sometimes you sort of get tired of the hand‑me‑down where only have the ports work, you would really like to have a switch that you can monitor the interfaces, things like that. And so we felt like it was only fair to repay the founders as well as be able to have enough money to make a sustainable future. And we did some ‑‑ some members paid no problem, and a few members left and some other members sort of keep for getting to pay the bill.

CHAIR: More questions, please?

SPEAKER: Hi, Blake. Thanks for putting this together. Very helpful. Speaking as a large multinational carrier, the things I look at when I am putting together a list of exchanging I am trying to connect to is I am basically forced to look at two things: One is the amount of outbacks it's going to cost me and the amount of value that business exchange is going to provide for me and the shortlist is what can I do with this, I am allowed to private sales, closer used groups, is there an SLA? And some of the other things I could mention is, perhaps in more flexible payment model might actually help companies like Zayo for example, laser focused on reducing opex all over the place and if I could buy toilet paper with an IRU, I would. And another suggestion is, as a provider of co‑location, if it was maybe easier for me to understand what it would require for an exchange to put a node into one of my co‑los and what your requirements are, that would make it easier for me to move forward with that sort of thing. Thanks. Orb orb for example, we have five nodes Belgrade, two of them are bigger, where we keep our switches and our servers and it is running roughly one reg space with 3 .5, 4.5 kilowatts. Other smaller POPs are just one or two switches and it's like 200 watts. So, fibre between those POPs is something that should be taken into account.

ULF KIEBER: From our side, most of the fibre we use is actually sponsored by one or two members, so I don't really have a need for additional fibres right now.

WILL HARGRAVE: In terms of the co‑location space and thinking it's quite in‑depth topic, as an Internet Exchange operator thinking about when you might expand into new co‑location facilities and what the opportunities are. Obviously for a data centre operator it's a good ‑‑ it provides with you a good advantage to have an Internet Exchange or one or more Internet Exchanges in your data centre and when I look at them I am really interested in the kinds of customers that you are looking to attract into the data centre because if your space is filled up with people who don't have AS numbers then I can't give them anything so I can ‑‑ we can all shake hands and cuddle and I can put a switch in a data centre but it's not going to help anyone. So, I suppose the ‑‑ my response to that query is I am really looking at what you plan and want to achieve from the data centre space you are selling, will it become a true interconnection site, a true Nexus of interconnection because that is where the Internet Exchange could be useful.

CHAIR: Any more questions? We have time. Orb orb they are waiting for another question to come back to do we loose any customers ‑ never, not even one. So we were only growing growing growing and customers for some reason, once they connect to Internet Exchange, very rarely go away. Very rarely.

WILL HARGRAVE: This is our experience as well, a major reason why people leave is because companies merge ‑‑ mergers, and yes it's sort of sad to see a network go away but that is nearly always actually consolidation and the traffic still stays on the exchange. We just see members merge.

ULF KIEBER: We have seen a downgrade when we introduce port fees three years ago, but we haven't ‑‑ we haven't really seen people leave.

CHAIR: Any other questions?

If there are no other questions. Would you recommend people to start a community exchange and what would you recommend to those people who are thinking about that?

WILL HARGRAVE: I love community exchanges, I basically devote my life to running a not for profit and it's not mine to sell, I am not going to get rich and get a new car or house or anything like that. But if you are operator or similar in a city and there is no Internet Exchange there, you might consider if you need one, and the way to do that is to get interested parties around a table, in the pub or coffee or whatever your local method of social interaction; get people talking, get people want to go achieve this and working together to exchange traffic. The technical stuff comes later, really. Orb orb so communication, collaboration, cooperation. I cannot thank enough to Mr. Martin Hide, because Martin introduced me to many, many, many CDNs and route DNS server guys and many companies that are interested in connecting to Internet Exchange, then there is a cooperation of ISOC, ISOC helped us tremendously in the second year when it was ‑‑ the question whether we will continue or not, they donated a couple of switches, nice big Cisco switches with 10 G ports, first 10 G ports that, how to say, are routable in our network, before that it was just switches. So, thanks to Martin, thanks to ISOC, thanks to other Internet giants really starting an Internet Exchange without that, without collaboration, communication, cooperation, is almost impossible.

LESLIE CARR: Well, I would say if you want to start an Internet Exchange make sure that you have a number of other people that can help you because one of the biggest problems, at least in my experience, has just been time. So if you have other people or if you have a very accommodating day job as well, because, you know, it takes time.

ULF KIEBER: Yes and actually it's a lot more work than just the configuring the switchboard. I mean that is a tiny little part of the whole business.

CHAIR: And we have a question over there.


FILIZ YILMAZ: I have two questions, but we have time. So first one is, there seems to be some trend towards a lot of RTT services for many exchanges around nowadays that are beyond the core Internet Exchange services as we know about them. First of all, do you think this is helpful towards the industry, I mean itself? Or does it have any adverse effect towards the costs and etc.. and the second one is, how do you also see the situation level of IXs because now we have, compared to old days, now we have a lot. Do you think there will be a situation level ‑‑ saturation level at some point that not every IX will find enough business even if they are technically good and able to provide? Thank you.

ZORAN PEROVIC: Let me take on RTT. Yes, we see growth of traffic mainly because of RTT providers, not huge, for example, in Serbia, Netflix is not so popular, it's not localised, but then local cable operators launched their own RTT services and we see traffic growing every day, not huge; maybe some 30% year on year. But yes, it is influencing positively amount of traffic in the exchange.

We started with 10 G switches from the day one, so we just upgraded, upgraded, upgraded and we will not have any issue until we need to upgrade to 100 gig.

Second question was? Sorry. Saturation, yes. For example, we don't see saturation yet. It's slowed down but not saturated.

LESLIE CARR: Well, I think America is a little more unique of an exchange market because it's dominated by a large commercial provider who can be quite expensive, honestly, and so I feel like there is a very important role for community and nonprofit exchanges because I think it can help drive the cost down for everyone and allow local ISPs who are running on more of a budget to still be able to get a lot of the benefits of peering. Now we are starting to see in some cases where multiple non‑profits are going into the same market which is interesting. I feel like giving an answer in that case is just, is a very dangerous thing, right? It would be nice maybe if there is some more conversations between exchange providers, if many of them want to go into a market so that there is not a duplication of efforts.

WILL HARGRAVE: I think actually, I will jump in here, the problem with Internet Exchange providers is they don't actually want to to each other especially in America because that is anti‑competition law and that's the kind of thing they put you in prison for 20 years for, so I have no intention of moving to another market but if I was I would be very worried about any form of, to be seen to doing any form of collusion as a major concern. If I can answer Filiz's questions. Saturation, well LONAP is in London and one of the largest interconnection cities in the world so, London will certainly bear more than one Internet Exchange. Quite what happens around the rest of the country is unproven and really that depends on the architectures that other people have decided for their network, so I don't think we are saturated yet. I think your first question was about are there services the Internet Exchange can provide? Yes. We have 80 members who provide all kinds of services and this perhaps goes back to my earlier point about competing with them. I would never want to have a transport product between cities or something like that, like we have plenty of members you can buy that from. But then there are some good Internet projects Internet Exchanges can do and I am pleased to be a part of those but not major commercial efforts because it increases the risk and we are not here for that, we are here for one purpose, really.

LESLIE CARR: All right. And I believe we are out of time.

CHAIR: We are out of time. We have a question from chat and probably this is the last one.

SPEAKER: I am Michaela from RIPE NCC, I am doing chat monitoring so I have a question but I don't know his name and I am waiting still for him to tell me, or her. Donal ‑‑ is asking should universities students try to build their own uni to uni ISPs?

LESLIE CARR: I would say no. Instead, volunteer to help your local community exchange.

ULF KIEBER: That will make a lot of IXPs.

WILL HARGRAVE: They are quite boring, it's just some switches and most of the work is going out and talking to people. I recommend doing that.

ZORAN PEROVIC: I second that.

(Applause)

ONDREJ SURY: Let me welcome Hans Petter Holen from RIPE NCC talking about RIPE Chair selection.

HANS PETTER HOLEN: Thank you. As said, I am RIPE chair, I have nothing to do with the RIPE NCC, just to make that clear. I am going to talk a bit about how we present, select Chairs so if could I have the presentation on the screens there as well, I can see what I'm presenting. Thank you. So, what did we to in the beginning, going back to RIPE 1, how did I get selected, what was the reactions from that, a proposal on how to do this, reactions on that and the road ahead.

So basically, at RIPE 1 there were 14 participants, according to the minutes I found on the Net, and as it says here in the conclusion/remarks, all agree on the fact that there should be someone to monitor the progress of RIPE, Rob Blokzijl was appointed as volunteer. So that is how Rob became Chair of RIPE and he was that for 25 years. It's very easy when there are 14 people in a room to select a chair, somebody volunteers, if somebody else else volunteers you can discuss and agree. Today we are a bit bigger community than that how did I get selected? Some of you were there when the Internet was for Europe was handed over to me. It started when I stepped down from the NRO NC and I sort of indicated to people that I would be interested in doing other stuff in the RIPE, such as taking part in task forces or chairing a Working Group or something like that. Rob asked me if I would be interested in taking over as RIPE Chair and after thinking about this, I said yes. To start that process off, Rob went around in his way to talk to people and discussed this and suggest this to build consensus, and then he presented me to the plenary at RIPE 67 when he announced that he had planned to retire as a deputy Chair and then at the meeting ‑‑ one meeting later, I was appointed as Chair by Rob Blokzijl and Rob said he didn't want to spend the last months of his career to make a fancy process to appoint a Chair, so he asked me to come up with the proposal in a couple of years' time.

So, there was some reactions to this. Some of them were delivered this person with let's say, enthusiasm. Others were in the RIPE NCC consultation report where they go out and have a questionnaire and do interviews with members of the RIPE NCC and community members and it was quite clear that there were two comments there; one is that some community members think there should have been an election and others also think that there they should be involved.

So then I started to look at what processes do they have in place. Okay, the community set up RIPE NCC years ago and while we say that RIPE NCC is separate from RIPE, this was set up by the RIPE community and in the articles of association of RIPE NCC we have a way of electing the RIPE NCC board. So we have sort of been through the process earlier on thinking when we elect people we actually need an election process.

Well, we say that RIPE is more than the RIPE NCC members. That is true. But the principles of these process comes out of this community. So, in the '99 ICANN was set up and we were to select members to the NRO NC serving as the ICANN Address Council. And when Rob did that the first time the process was very simple: Get nominations, present the candidate and we did paper ballot with votes with the people in the room. So this was actually the sort of the first way of selecting this. At this meeting will the first time we do electronic vote for the NRO NC members and I will come back to that later on today.

We also have a Programme Committee at RIPE where we have introduced electronic voting and what we see is that the number of votes we get in electronic votings versus paper based voting is much higher and we have seen that in the RIPE NCC General Assembly when we went from paper to electronic, there is much larger participation. So if we want somebody to be selected as RIPE Chair with a large sort of vote of confidence, we should go from paper to electronic I think is a clear tendency here.

So, I put together a draft proposal that I published at the last RIPE meeting, very simple: Open call for nominations, require support from five members of the community so you can't just walk in, you need some references to put your name behind your candidacy. Present the candidates at the RIPE meeting, have an electronic open vote and oversight by a committee of trusted individuals.

Ah, but then there were some reactions to that as well. We don't do elections. It should be done by consensus. Okay, so how does 600 people at the RIPE meeting select a Chair by consensus? That is an interesting one. I would like to see that presented if somebody can come up with the consensus‑based process, yes. You can Google it and find there are consensus‑based voting mechanisms invented over time, quite interesting readings. I like to keep things simple, though. So, in this discussion there was also some but what is the role, the what is the RIPE Chair doing, so Mirjam has been kind enough to put together an updated function description of what the RIPE Chair has been doing so that is in a draft form published. We have set up a mailing list for the RIPE Chair discuss and that has been published on the RIPE list. There has started some discussion there already but you are all welcome to engage and subscribe to this list in order to shape this process. And what we have in front of us is actually some key questions, like what are the requirements or responsibilities for this person, there is a draft there, we need to refine and conclude on that. Who can be selected as RIPE chair? Who will select the RIPE Chair and how will this election happen? And that is what I have prepared and then I would like the session moderators to open the floor for suggestions from the the community.

ONDREJ SURY: Can we have the light.

FILIZ YILMAZ: I will start with repeating what I kind of said, I think I am one of those people who has been ‑‑ you have been pointing that there was some noise in the RIPE Chair discuss. My personal opinion is that I am not afraid of an election and I think election might be a good tool to choose our leader. It is ‑‑ it can provide an open system where other candidates will be welcome because in a consensus‑based decision‑making selection process, the people who are not incumbent of positions can be often disadvantaged because it's hard to race against a strong person, even if you are good and confident and willing to do so. And it can be awkward and all. There is an amenity element in an election system where if you are not supporting candidate you don't have to show it publically, which saves space and everything. But apart from that election part, I do actually ‑‑ I want to thank you again that you brought this up. I have read the job description, Mirjam pointed. I think it's comprehensive enough. It doesn't need to be changed drastically in my opinion at all, but what I would suggest we make one proposal that includes all these things in one place. At the moment, things are linked from there, it is discussed in this mailing list. I even heard some people are not even aware of the RIPE Chair discuss mailing lists, that is why maybe there is less participation at the moment, they just don't know where it's happening. And finally, I am all with the transparent side and documentation side. I think once we establish these it's not too much work on our side to come to an agreement. I would be just very wary about trying to make consensus‑based selection system because this is not ‑‑ we are not talking about a proposal. Consensus works very well about the ideas, because you can shape and change the core of a proposal along a period of discussion. While you are talking about individuals who can do a job or not, you can't change the person and I think election would be more suitable in terms of the process here. Thank you.

ONDREJ SURY: So any more comments?

JIM REID: Just another bozo in this bus. I have to disagree quite strongly with everything Filiz has just said. I strongly strongly urge you not to use elections because any election mechanism is bound to be gained. We have an open forum here in the RIPE meeting, there is no membership criteria for being at RIPE meetings or part of the RIPE community and what worries me about having any election for the RIPE Chair is we could have a very unsuitable candidate come forward and manufacture 1,000, 5,000, 10,000 of the new Facebook friends or whatever it is and then rig the vote or stack the deck to get unsuitable candidate elected. If user consensus based mechanism we can use other techniques to try and finesse that and stop unsatisfactory candidates coming forward. We use consensus to make policy decisions, pretty much all the Working Groups as far as I know, use consensus to select the Working Group Chairs and use it throughout the history of RIPE for everything that really is important, and I think we should stick with that if we possibly can.

I would like to throw out another suggestion as well: Why don't we just do the same thing was done the last time when Rob nominated his successor and Hans Petter, if you ever get to the point of standing down why don't you nominate your successor? Second and final point on this is we discussed this in the Working Group Chairs' lunch last time and I think this is something might need other consideration by this group is, should there be a vice‑chair at some point to help with the success plan as well so that a new chairman or chairwoman, when they come forward, get time to come up to speed with all the stuff that is going on and might ease some of your workload, I don't know if this is taking up too much of your time and maybe there is someone else who can share that. That might be a good thing.

PETER KOCH: I think I need to disagree with almost everything that Jim said recently. And no seriously, supporting Filiz for the reason that I seriously believe that, A, this process needs some credibility and accountability, we have been talking about that yesterday; second is, the ad hoc mob style Facebook style plus one consensus finding for the Working Group Chairs is a total failure and I do not want to see this repeated for the RIPE Chair, and anything that would be ‑‑ any opportunity for gaming can be taken care of by defining the electorate and then putting some requirements on the candidates so that can be dealt with but we need to make sure that we have a credible process here and a good Chair, of course.

SPEAKER: Thank you. Malcolm Hutty. I would like to agree with what Filiz said earlier and particularly the last point. I am strongly opposed to voting for policy development. There are not a finite range of options and if there is not consensus then continuing the discussion to seek new options that could build consensus is the right thing to do and sometimes if you can't build consensus that is a very clear signal that doing nothing is better. Electing a Chair is quite a different thing with completely different considerations and we need to have somebody and you can't change those people, you have to choose between them so they are both completely different considerations, for those reasons I am not afraid of electing the Chair. I acknowledge the point Jim made by stacking the electorate, particularly in a community at that has an ill‑defined or undefined membership, I don't view that as necessarily unsoluble problem, we can manage that risk to acceptable level, thank you.

ONDREJ SURY: Okay, I have a comment as well, then the gentleman in blue. Just, there are organisations that have a Nom Com mechanism, can might be an option as well.

HANS PETTER HOLEN: I am Chair of the ICANN Nomcom this year and that is an interesting construction, around 20 people from the community, which is, has defined process to call for nominees and then that committee uses a process of its own choosing by going through the candidates and selecting one of them and confirming that with majority vote at the end. So usually the committee is unanimous when they select their candidates. So it's a ‑‑ it's a consensus‑based process within what some people view as a black box because they can't see what is going on within the Nomcom and can't influence the results. It has pros and cons but definitely an option.

ONDREJ SURY: Do you think it would be good for RIPE Chair selection as an option or does it work well for ICANN?

HANS PETTER HOLEN: Well as I said, some people view it as a black box and are very critical of the construction of a Nomcom at all. It doesn't really matter what I personally think. I think what we are looking for is something that the community is comfortable with, and I kind of hear two strong voices from where I am standing. It's those who think we still can work the procedures we did when we were 14 around the table, that scaled up to maybe 140, and it's those who actually look at what is happening in the world around us and we need more accountability and then well‑defined processes like election is the way to go. And I sympathise for both so I'm kind of right now in the position where I just want to take a step back and have a look at what does the community want here and then try to summarise that in the end.

SPEAKER: Hello ‑‑ I actually have a question about how ‑‑ how long a Chair ‑‑ what is the turnover of Chairs? Is it like Rob, almost lifetime or five years or ten years? I mean, if it's a lifetime job, which I would expect another Chair in next 20 years, is it a bit too early to start discussing procedures if that thing actually is going to happen two years' later? That is the first thing. And second thing is, sometime back we had a members discussion to when a lot of people didn't figure we actually have that list want to become member. RIPE member and RIPE community have grown in size and it's ‑‑ the scale of the RIPE community and of the membership is totally different from even five years ago when I started, so we do need a more inclusive method and less complicated way to include everybody, not only people in the meeting because the Chair of RIPE also influence every ISP, everybody in the Internet, so it should be even more inclusive and more open and more reach out than just about in a meeting just for the people who attended a meeting, no matter it's ‑‑ some other mechanism but it should be more inclusive, thank you very much.

RUEDIGER VOLK: Deutsche Telekom. First, let me way there was one point of Jim that I don't disagree with, and that is putting, well okay, giving the Chair, vice‑chair I think is something very reasonable. Else, I don't see adequate defined procedures to run a valid consensus process for this, with the whole community, and Nomcom process like in the IETF actually would be as Hans Petter has pointed out, actually a consensus process, and in the IETF I think the Nomcom is working pretty successfully and is, I think, very well accepted. Of course, there always are some flaws but well, it works pretty well. I am not sure what the differences between IETF and ICANN there are. However, I am pretty sure that the environment of the IETF allows their Nomcom process and mapping that process exactly with the random selection of candidates for the committee will not map on to RIPE. So, essentially, elections remain open as a way forward, but there quite obviously the question of how to define the electorate is very important and, well, kind of providing a process so that the actual candidates get proper exposure, I think also come with some challenges. So quite clearly, unspecific, very inclusive consensus process I think does not cut it quite clearly.

SPEAKER: Thank you.

NURANI NIMPUNO: I wanted to disagree as well with people, because ‑‑ but then some people said some very sensible things. So first of all, you know, we love consensus, consensus has worked really well in this community but let's not confuse the terms here. Consensus is a decision‑making mechanism to shape good policy so we always said we don't want Hans Petter to give a proposal and then everyone votes. The idea is we, together, reason about what is ‑‑ how you make a good policy. And I think that is very different from selecting a Chair. So, consensus certainly has its place in when the community makes joint decisions about policy and other things. So let's keep it there. But I don't think that necessarily means it's appropriate for selecting a Chair. I think elections, I don't think we should be scared of elections to ‑‑ for selecting the RIPE Chair.

I think in Nomcom version is also very sensible way of doing it. That way you get trusted community members who reason around nominees and then together make a good decision.

And then ‑‑ and I also want to agree with some of the comments made by Peter Koch and Filiz as well. You know, we had discussion about accountability before having clear procedures for these things will actually strengthen our accountability. I think the job description that Mirjam wrote was a very good one, so I also think that before we go too crazy bananas when discussing how to select a RIPE Chair, let's be aware of keep in mind of what the role is. It is not the representative of all ISPs in the world; it has a very specific role, so we don't need to over‑engineer that process. But then I had a tiny, well a detailed comment as well, and I think maybe in the ‑‑ in the requirements there should be something about having been an active member in the RIPE community for X amount of years because as a RIPE Chair you need to understand community and that might also reduce that risk that Jim was talking about of having someone fly into a meeting and get all their Facebook friends to vote for that person just for that particular meeting.

So, I guess in the end, I agreed with a lot of sensible comments. But I don't agree with some of the comments Jim made.

JOAO DAMAS: No if I filliation for this one. I would like to share my support for the election‑like process and I am stating I am quite afraid of things like what Jim just said. Being afraid of elections because you are not in control of the outcome, is a sort of speech we hear more often in places these days. It makes me really scared. The other thing I want to say was related to Nomcoms. They are a little bit weird. I have been participating in both the ICANN and the IETF ones. I wouldn't call them consensus bodies at all, they are more negotiation than consensus because there is more than position to be elected, there is this trading game goes on all the time, give me this position for this person I support and in exchange two positions for this other body. So it is not the same. Having said that, you can ‑‑ clearly need to define an electorate and that is actually how you control the elections. And that is it.

SPEAKER: Wilhelm Boeddinghaus. I feel that elections will show that our community is not one community any more. We speak of ISP and enterprise and totally different interests in what they want and the direction they want the RIPE to go. We have those who support this community, or have supported this community for the last 20 years and we have those who just joined the communities to make money out of the rest of IPv4. So we have different groups and I hope that if we elect a Chair, that we don't get into dog fights before, for the best candidate to steer the RIPE in one or the other direction. Don't forget that the RIPE does a lot of consulting to governments and we have so different governments in our community like Saudi Arabia, Switzerland, Russia, Norway, so totally different interests from their side so they might even try to influence an election by just sending a lot of members, community members to our meetings just to influence the election. So I have no better idea than an election but I think it's not a very good process because others might try to influence where our community is going, is heading.

SPEAKER: Hello, Daniel Karrenberg. For the purpose of this I am the only ex RIPE Vice‑Chair in existence. Not many people remember this. So, first of all, I think the Vice‑Chair idea is a very good one but the scope of that position should be just to advise the Chair, no other powers or job description, and there should be a very strong connotation that the Vice‑Chair is the intended next Chair. That was one.

Number two: I agree with everything that Jim Reid said, and I think it's quite unfair to characterise it as controlling the elections. I see it more as defining a procedure that yields the selection of a person that is actually good for the job. If you do an election, you will get things that the last speaker said, like, you know, factionising, you will get politics and you will elect the person that is most popular or that achieves a successful in rigging the election. If you have the election procedure that the Chair actually selects the next Chair, you will have a much higher possibility to select a person that is actually fit for the job. We have been successful actually so far in doing this with the Working Group Chairs, I disagree with Peter there; quite successful, and we have also been successful, I might remind people, to remove Working Group Chairs that were no longer, no longer had the support of the Working Group and the community and we didn't need a procedure for that either. So, I am fully and squarely against elections, not for any other reason but that we will not ‑‑ elections will not yield the best person for the job. Thank you.

CHAIR: Okay, colleagues, we have definitely run out of time and run out of the coffee ‑‑ so, we can continue our discussion during the coffee break right now. And the session is closed. Thank you very much.
(Applause)