I will be teaching the SANS Security Essentials (Security 401) class at the University of New Mexico August 11-16. We'll be doing it bootcamp-style (yeah, baby!), so sign up if you want to work hard, play hard, and meet some other security geeks.
SANS Community Albuquerque 2008
Sunday, June 29, 2008
Friday, June 20, 2008
Little Things like Butterfly Wings
Little things count. They count a lot. In fact, attention to the little things... the details... and how they fit together makes it easy to bring the big picture to realization.
There are so many times that I have experienced large system failures because someone was too busy, too tired, too lazy, too rushed, too something to do the job correctly. The funny thing is... it is always more work to go back and fix what wasn't done right the first time than it would have to have done the job right the first time. Intuitively people seem to understand this, but why don't they DO it?
Recently I had a network meltdown because a developer on was building a network application. This happened behind a firewall, which is an interesting story for another time. Even though the network was "isolated" we experienced the meltdown because the developer glossed over some "little things" ... and so did I.
His mistake was deciding that on this isolated little test network, he didn't need to follow the recommendations in the RFC -- required by the test network switch he was using -- to re-send ARP packets to reset the aging table timeout. This had a fascinating effect on the test network that was only illuminated by watch the switch port traffic. For a short, initial period of time, the switch would dutifully forward packets from the port the test generator was plugged into to the port the receiving test system was connected to. Then, after the timeout occurred, and no new ARP packet was seen, the switch happily turned itself into a dumb hub... dutifully forwarding packets to every port with link (including the uplink to the switch where the firewall was located).
My mistake was believing that I had achieved isolation by routing a VLAN though a firewall and then back into the same switch as the production network. I convinced myself that I was concerned with stability, not clever attackers or malware... and therefore became complacent.
The engineer is fixing his problem (surely customers would not appreciate this test bench behavior). And I have learned my lesson as well.
The little things count -- just like butterfly wings.
--john
There are so many times that I have experienced large system failures because someone was too busy, too tired, too lazy, too rushed, too something to do the job correctly. The funny thing is... it is always more work to go back and fix what wasn't done right the first time than it would have to have done the job right the first time. Intuitively people seem to understand this, but why don't they DO it?
Recently I had a network meltdown because a developer on was building a network application. This happened behind a firewall, which is an interesting story for another time. Even though the network was "isolated" we experienced the meltdown because the developer glossed over some "little things" ... and so did I.
His mistake was deciding that on this isolated little test network, he didn't need to follow the recommendations in the RFC -- required by the test network switch he was using -- to re-send ARP packets to reset the aging table timeout. This had a fascinating effect on the test network that was only illuminated by watch the switch port traffic. For a short, initial period of time, the switch would dutifully forward packets from the port the test generator was plugged into to the port the receiving test system was connected to. Then, after the timeout occurred, and no new ARP packet was seen, the switch happily turned itself into a dumb hub... dutifully forwarding packets to every port with link (including the uplink to the switch where the firewall was located).
My mistake was believing that I had achieved isolation by routing a VLAN though a firewall and then back into the same switch as the production network. I convinced myself that I was concerned with stability, not clever attackers or malware... and therefore became complacent.
The engineer is fixing his problem (surely customers would not appreciate this test bench behavior). And I have learned my lesson as well.
The little things count -- just like butterfly wings.
--john
Subscribe to:
Posts (Atom)
