Scanning the internet in under 45 minutes

By Curtis Collicutt, Cloud Developer, Edmonton

How big is the internet? It depends on how you define size… Is it simply the total IPv4 address space (approximately 2^32), or is it the number of hosts or devices attached? Or some other measure? Regardless, it's safe to say the internet is a huge, global system with millions of individual networks and nodes.

It has always been difficult to gauge the size, shape, and connectivity of the internet, as the only way to do that is to (slowly) scan the entire IPv4 address space, which can take days or even weeks. But now, researchers at the University of Michigan have released a tool called zmap which can scan the entire IPv4 address space in under 45 minutes using only a small server (with 4GB of RAM!) and a gigabit ethernet connection '€” tools that are available to almost any researcher.

In their paper, ZMap: Fast Internet-Wide Scanning and its Security Applications, which was recently delivered at the 22nd USENIX security symposium, the researchers detail the unique design of zmap. They also discuss the results from various internet-wide scans they have performed, and give some pointers on how to complete an internet-wide scan while still being a '€œgood internet citizen.'€

Matrix

The paper compares zmap to one of the most well known network scanning tools, nmap (which is used in the The Matrix as part of a '€œhacking'€ session, pictured above), and shows that zmap can operate 1,300 times faster than nmap, even when nmap is configured with its '€œinsane'€ setting. Part of the reason that zmap can scan so quickly is that it keeps no per-connection state, instead using the novel idea of storing scanned node details in the returned packets themselves.

The paper discusses several zmap applications and security considerations:

  • Tracking protocol adoption (such as HTTPS)
  • Enumerating Internet-wide vulnerabilities (such as UPnP and IPMI/BMC security concerns)
  • Discovering unadvertised services
  • Fingerprinting hosts to '€œfollow'€ travelers
  • Monitoring service availability (the researchers scanned the Internet during Hurricane Sandy)

The researchers conclude that:

We are living in a unique period in the history of the internet: typical office networks are becoming fast enough to exhaustively scan the IPv4 address space, yet IPv6 (with its much larger address space) has not yet been widely deployed.

While it may be years, or even decades, before IPv6 is completely deployed, the sheer size of the IPv6 address space will make it extremely time consuming, if not impossible, to scan it all.

Until then, with the development of the zmap tool, it looks like the entire internet can now be scanned in under an hour, and that brings with it many interesting research possibilities, as wells as illuminating concerns regarding high-speed mass server infections.