Litui.Net

The company I work for, Cansel Survey Equipment runs a network of Trimble GPS base stations across Canada in multiple VRS configurations. This network (Can-Net) is powered on the backend by Trimble Infrastructure software. I’ve been integrating around this software for about 2 years now.

Integration and Abstraction

While performing admirably at its intended job (many of the details of which still elude me as I’m no geomatics engineer) the software can be a little monolithic. It works as advertised, make no mistake, but its UI is a bit clunky and it doesn’t seem to integrate well into other applications without a great deal of massaging. [Please Note: I say this with respect for the fact that the software is easily the best available for GPS/GNSS infrastructure and it does seem as though Trimble is making strides towards modernizing their applications.]

One of my main goals with regard to the infrastructure software has been (while working within and around the known limitations of the software) to abstract account management, reporting, and our internal tech support processes from staff access to the actual servers themselves. There are many reasons why this is necessary in an operation this size:

Automation

It can be very difficult for staff to manage sales and account terms in ERP software alongside user accounts in the GPS Infrastructure. Automation is necessary to ensure that customers are getting the accounts and access they paid for and freeloaders are kept out.

Accountability

Having user access to the database abstracted through a custom management tool means a greater level of logging of historic changes to user access tables and moderation of the types of changes that can be made.

Stability

More staff accessing the servers means a greater possibility of staff accidentally closing the server applications (much of the software runs windowed) and making problematic changes. These sorts of things can interrupt end-user access. Abstraction means greater regulation of what software runs on the servers and the resources used by that software.

Security

Giving direct access to staff invites unintended use. Since simply opening a web browser to a questionable page in Windows can result in virus/malware infection it’s important to keep the list of users with access to the actual servers to a minimum.

This necessity resulted in the ongoing programming of an intranet web application (written primarily in Ruby on Rails) which I may discuss in more detail in another post.

Bolt-ons

One of the roadblocks in the abstraction process has been the lack of useful real-time output from the infrastructure software for monitoring purposes. Access logging occurs only after a user logs off. This has made it necessary for our technical support to call the server admins to obtain information on whether or not a station is streaming data or a user is online. This is not ideal and for reasons stated above, it’s even less ideal to give them server access simply to check the online users list.

Bolt-on #1: Google Maps

While not my idea, this was a good choice for (near) real-time stationary station connectivity status.  Using Google Maps allows not only our staff, but all users and the general public (required by Google’s terms of use) the ability to check our station connectivity status.

My implementation uses protocols available in the infrastructure software to obtain a list of stations and logs that to a static file on a periodic basis. This allows me to place the file off the servers (better for security) as well as to provide a very quick-loading source of data for the Google map (the server’s own protocols for obtaining this data are insanely slow).

I have a plan to implement more up-to-date station statistics to our tech support staff using a means other than Google Maps, but more on that another time.

Bolt-on #2: Who’s Online?

As mentioned earlier, getting a list of the users currently connected to our system requires access to the server. In particular, access to the User Interface software for Trimble’s NTRIP Caster service. The UI provides usernames of connected users, amounts of data transferred, mountpoints (data streams), source IP addresses and source ports. Unfortunately, owing to the fact that many of our accounts have serialized alphanumeric usernames (an important part of our ERP integration) the username itself doesn’t tell us much about who is connected. Ideally, we need an approach that allows us to cross-reference the usernames with our ERP to provide customer associations and to view the list of users via a management tool running on a separate machine.

I’ve heard of some administrators using screen scraping utilities to provide this data to their customers. This method would require the GUI to be maximized and foremost on the screen at all times. Additionally, the data would be an image (unsortable, unfilterable and unusable for anything but display).

My method requires a little more diligence but results in the type of data and access we need:

The most important part of this method is the tool TShark, part of the Wireshark package. TShark and Wireshark are network packet analysis tools. They (with the help of passive packet capturing tools) can apply filters to network traffic in realtime and output specific information about the data. For this purpose, I set up TShark to run realtime analysis on the same system running the NTRIP Caster, monitor the same ports, filter out HTTP Authentication data (NTRIP is based on HTTP and uses basic auth) on those ports and produce log files of selected fields (namely: source IP, source port, and username/password).

I then wrote a script on a separate server that obtains these authentication log files, uses SNMP to obtain a list of source IPs and ports for current connections to the NTRIP caster, compares the IPs and ports to those in the log files, and loads the most recent matching usernames into an array. It then does some database magic to match the usernames to customers and outputs the result as a JSON file which can be parsed using libraries available in most programming languages. This script is run by Cron every minute.

Using an AJAX intranet web page that reloads and parses the JSON file every minute, I can provide a list of current users to our technical support staff that’s internal to our company network and does not require direct access to the servers.

[Edit: It should be noted that TShark chews up RAM pretty quickly.  I have it restarting on a daily basis so it doesn't eat all the system memory.  There are probably other tools that can do the job, or you could write a custom parser that's more efficient if that's a concern.  I chose TShark because it was the easiest solution to implement out-of-the-box.]

Oh, hello again blog.  It’s been awhile, hasn’t it?  I know I’ve been neglecting you and I can’t promise to be any better about it, but I’ll feed you some scraps and hopefully that will tide you over for awhile.

While writing this I’ve been obsessively pressing F12 to check on my UPS shipment of a Samsung NC10 netbook [update: just now arrived!  yummy!].  I’ll likely be installing an alternative OS of some sort on it.  Probably Ubuntu Linux, but if I decide to change out the wireless adapter I may take a stab at turning it into a Hackintosh.  I recently spent some time (and factually, am still spending time) converting my desktop system, Butlertron, over to OS X.  Miraculously, it mostly works with only a few issues (occasional kernel panics on SATA controller).  It reminds me of all the effort I had to put into getting Linux working on my hardware back in the late ’90s.

Work at Cansel continues to go swimmingly.  I’m getting closer to the end of my third year of employment here (July) and looking back I can’t believe how much I’ve learned and accomplished here so far.  It’s not all fun and games, but it’s often stimulating and I feel the work I do is appreciated.  Work is still sending me traveling quite frequently.  In December, my boss and I traveled together to both Vancouver and Toronto to bring Global CADD Systems into the company fold.  Next week I’ll be going to St. John’s, Newfoundland to visit our new branch at MF Kelly.  Later in the week, I’ll be visiting our Halifax (Dartmouth) branch.  I also have several major projects due within a month.  Busy times.

I’ve been doing a lot of programming lately and I’m actually starting to feel like a programmer.  Those who know me will likely be rolling their eyes at this point, but it’s true that I rarely consider myself a programmer by nature.  I still feel insecure about the quality of my programming, but it’s finally getting to the point where I can pretty well sit down and write up whatever I need for the purpose in any of a handful of languages.  That’s a pretty nice feeling.

I continue to swing dance and will be in attendance at Frankie Manning’s 95th Birthday Party in New York in May.  This will be my first trip to New York as well so expect some crappy pics of Lady Liberty and the Empire State Building to surface a couple months from now.  In the more immediate future I’ll be attending 9PLEX A.K.A.: PLEX and the City here in Calgary.  This coming Friday and Saturday nights there will be dances with the Bow Djangos and the Polyjesters, respectively.  Both are very fast bands and I’ve been working hard at improving my Balboa accordingly.

This year I also introduced my girlfriend to Lindy Hop and she seems to be enjoying herself though she lacks the time in her busy school schedule to take dance classes at present.  Oh yes, I have a girlfriend named Jodie who makes me very happy. ♥