For some reason I felt compelled to define the word "busy":
*Raising a 5 year old son
*Working an average of 55 hours per week
*Going to college
*Being married to someone who is working two jobs
It was a nice weekend. Saturday Eno had therapy, swimming lessons, and a birthday party. The birthday party was at a tennis park in Winnetka so we both got sun burnt. ON the way home we stopped at best buy and got one of the dance pads you can connect to your play station. Man those are some hard games. I totally sucked at it, but I'm sure you would have all predicted that anyway. Sunday I spent about 5 hours in the library reading. I have about 200 pages to get through on a daily basis for this class. I'm mainly reading the Federalist papers, constitution, and the framing of it. It's amazing how someone like Bush could have become president compared to someone like James Madison!
I finally got our CallManagers upgraded Friday. Weee.. going to NY sometime this week to install network equipment for our new WAN connection. We are getting dedicated 6MB between all of our offices.
We were eating dinner last night with Larry and his family. Around 5pm I noticed a firetruck and ambulance in front of my home. Naturally I ran out to see what happened. Our neighbors daughter tripped while running and bumped her head on the sidewalk. Nothing to serious. Lots of blood. I came back to my pizza. About ten minutes later an ambulance screeches by the house. I immediately thought maybe the little girl passed out and ran outside again. No, she was fine and the ambulance kept going down the street. At the end of my block were two police officers talking with a neighbor. I approached them and asked what was going on. They explained that someone had forced their way into a ladies home down the block and stabbed her. That was who was in the ambulance that just blew by. After some more questioning it turns out Lonnie may have had en encounter with the suspect. About a month ago someone came up to our house. Kind of raggedy, dirty, ripped clothes, smelly, missing tooth. He explained that he just moved to the neighborhood and was our neighbor. (Lonnie or I have never seen him.) Said his wife's car go towed and he needed ten or twenty dollars. He asked if he could come in and do some handyman work around the house. Lonnie said no. A few minutes later she saw him sitting in a pickup truck in the alley staring in to our house and immediately called 911. After talking to the police yesterday about the stabbing they explained a person of this description had been going from house to house in our neighborhood giving the same story for about a month. Whats his motive, why force your way in on an old lady, stab her and run away on foot? Hopefully she is OK. Last I heard she was in surgery. It's also pretty bold to do this during the day when people are around. She was assaulted around 5:15pm. There were firetrucks and ambulances one block down the street from the scene! Obviously he did't care. I think I am going to devise a siren on top of the garage with a hidden switch behind the front door. If anyone tries to forcefully enter the house Lonnie, Eno or I can just flip the switch and set off the loudest damn siren you ever heard.
So I was watching a Tivo'd version of Dr. Who on the SciFi on Friday. During one of the commercials I noticed a very quick flash and was sure that I had seen a face. So I paused the Tivo, rewound about 2 seconds and proceeded to move forward frame by frame. After about thirty or so frames I got the following image on my screen:
Just one frame and then back to normal. Nurses covered in blood with nasty scary teeth! What the hell! I still have it saved on my Tivo. Has anyone else seen anything like this on SciFi?
It was a crazy weekend. My CallManager/BGP network upgrades got canceled because we were having connectivity problems to remote sites and we didn't want to introduce any more changes until all the problems got resolved. We started getting a bunch of complaints that users could use email and that messages would just sit in users outboxes. Outlook would eventually just pop up one of its "Trying to connect" messages balloons down by the task bar. We proceeded to troubleshoot the issue to our San Francisco office and New York offices and discovered that any packet with a bytes size larger the 128 bytes was getting dropped. Both of our office are in UBS office buildings and UBS provides all of the network equipment and connectivity. So trouble shooting from their end was very difficult. Also both sites are connected back to headquarters via a IPSEC connection. In the end we discovered that if we installed the Cisco VPN client on the end users machines and used that to connect instead of the L2L tunnel everything worked fine. So today we are chasing after UBS to try and trouble shoot their VPN concentrator which terminates the remote end of the L2L tunnel.
Also on Saturday we were installing new A/C units in our data center and relocating network equipment to make room for two new 6500 core switches. During that process many of the machines in the data center lost connectivity for short periods of time. After the move was completed Nagios started screaming that half the network was down. I immediately thought this was a problem with Nagios or the Nagios box itself because everything else seemed fine. I spent about four hours playing with duplex settings, speed settings, kernel drivers, etc on the Nagios box to try and figure out what the problem was. Then I started to notice that I was seeing issues outside of Nagios. If we attempted to ping any host within our production subnet we experienced about 40% - 80% packet loss. Nagios wasn't't broke! It was doing exactly what it was supposed to do! It was telling me the network was f&8^ked up! Long story short — we narrowed the problem down to a single bad SFP GBIC in one of the IDF closets.
I have an interesting weekend coming. Starting Friday around 6pm we are going to be migrating from 3 bonded T1s a fully redundant BGP Internet connection. I finally get to play with BGP!
The new setup will consist of 3 T1 lines from MCI, 2 T1 lines from SBC, and a 10MB DS3 connection from Yipes. So starting at 6pm we will be modifying all our NAT rules on our external facing PIX firewalls, modifying DNS entries, modifying our internal routes and crossing our fingers. Should be fun?
At the same time I’m going to be upgrading our Call Managers to version 4.1.3 sr3a. The OS to version 4.2 and the IOS on our voice gateway. We have been troubleshooting an intermittent problem where inbound calls transferred get dropped. Sr3a seems to have dozens of fixes related to call transfers.
Another interesting item:
Our developers are working on a program that takes in a stream of pricing data, modifies it and then sends the data out to users that have subscribed to it. Everything is written in C++ and compiled with gcc. All of this happens in real-time and each unicast session takes about 2.1Mbs. That’s a lot of data! We have 100 traders running clients that talk to this thing. That’s over 200Mbs of data. I’m keeping my fingers crossed on this one. This is all running on a RHAS 4 Dell box with 4×2.8Ghz dual core xeon CPUs. In testing mode we are processing about 4000 prices per second and the load on the box is only .01! The network usage has spiked as high as 40Mbs. I’m wondering if this thing is going to be able to sustain 200Mbs.
We are using this really, really cool tool called NexVu to monitor our network traffic. It’s a dell box running Linux and their proprietary application. Check it out. http://www.nexvu.com
My new cat. She was a stray off the street. 1 year old. Very sweet. Seems to get along with Cosmo. She is now fixed, got all her shots and a RFID tag. Thoughts on a name for her?
No it wasn't. I moved the server over to the T1 line a couple of days ago and Easydns didn't reload my zone files. So my primary namerver had the correct IP and the secondary nameservers had the old IP. I had to force the Easydns servers to perform a reload of the zones. What kind of crap is that!
Anyone interestead in some co-locaiton space? Know anyone that is? I'v got cheap bandwidth available.