Litui.Net

The company I work for, Cansel Survey Equipment runs a network of Trimble GPS base stations across Canada in multiple VRS configurations. This network (Can-Net) is powered on the backend by Trimble Infrastructure software. I’ve been integrating around this software for about 2 years now.

Integration and Abstraction

While performing admirably at its intended job (many of the details of which still elude me as I’m no geomatics engineer) the software can be a little monolithic. It works as advertised, make no mistake, but its UI is a bit clunky and it doesn’t seem to integrate well into other applications without a great deal of massaging. [Please Note: I say this with respect for the fact that the software is easily the best available for GPS/GNSS infrastructure and it does seem as though Trimble is making strides towards modernizing their applications.]

One of my main goals with regard to the infrastructure software has been (while working within and around the known limitations of the software) to abstract account management, reporting, and our internal tech support processes from staff access to the actual servers themselves. There are many reasons why this is necessary in an operation this size:

Automation

It can be very difficult for staff to manage sales and account terms in ERP software alongside user accounts in the GPS Infrastructure. Automation is necessary to ensure that customers are getting the accounts and access they paid for and freeloaders are kept out.

Accountability

Having user access to the database abstracted through a custom management tool means a greater level of logging of historic changes to user access tables and moderation of the types of changes that can be made.

Stability

More staff accessing the servers means a greater possibility of staff accidentally closing the server applications (much of the software runs windowed) and making problematic changes. These sorts of things can interrupt end-user access. Abstraction means greater regulation of what software runs on the servers and the resources used by that software.

Security

Giving direct access to staff invites unintended use. Since simply opening a web browser to a questionable page in Windows can result in virus/malware infection it’s important to keep the list of users with access to the actual servers to a minimum.

This necessity resulted in the ongoing programming of an intranet web application (written primarily in Ruby on Rails) which I may discuss in more detail in another post.

Bolt-ons

One of the roadblocks in the abstraction process has been the lack of useful real-time output from the infrastructure software for monitoring purposes. Access logging occurs only after a user logs off. This has made it necessary for our technical support to call the server admins to obtain information on whether or not a station is streaming data or a user is online. This is not ideal and for reasons stated above, it’s even less ideal to give them server access simply to check the online users list.

Bolt-on #1: Google Maps

While not my idea, this was a good choice for (near) real-time stationary station connectivity status.  Using Google Maps allows not only our staff, but all users and the general public (required by Google’s terms of use) the ability to check our station connectivity status.

My implementation uses protocols available in the infrastructure software to obtain a list of stations and logs that to a static file on a periodic basis. This allows me to place the file off the servers (better for security) as well as to provide a very quick-loading source of data for the Google map (the server’s own protocols for obtaining this data are insanely slow).

I have a plan to implement more up-to-date station statistics to our tech support staff using a means other than Google Maps, but more on that another time.

Bolt-on #2: Who’s Online?

As mentioned earlier, getting a list of the users currently connected to our system requires access to the server. In particular, access to the User Interface software for Trimble’s NTRIP Caster service. The UI provides usernames of connected users, amounts of data transferred, mountpoints (data streams), source IP addresses and source ports. Unfortunately, owing to the fact that many of our accounts have serialized alphanumeric usernames (an important part of our ERP integration) the username itself doesn’t tell us much about who is connected. Ideally, we need an approach that allows us to cross-reference the usernames with our ERP to provide customer associations and to view the list of users via a management tool running on a separate machine.

I’ve heard of some administrators using screen scraping utilities to provide this data to their customers. This method would require the GUI to be maximized and foremost on the screen at all times. Additionally, the data would be an image (unsortable, unfilterable and unusable for anything but display).

My method requires a little more diligence but results in the type of data and access we need:

The most important part of this method is the tool TShark, part of the Wireshark package. TShark and Wireshark are network packet analysis tools. They (with the help of passive packet capturing tools) can apply filters to network traffic in realtime and output specific information about the data. For this purpose, I set up TShark to run realtime analysis on the same system running the NTRIP Caster, monitor the same ports, filter out HTTP Authentication data (NTRIP is based on HTTP and uses basic auth) on those ports and produce log files of selected fields (namely: source IP, source port, and username/password).

I then wrote a script on a separate server that obtains these authentication log files, uses SNMP to obtain a list of source IPs and ports for current connections to the NTRIP caster, compares the IPs and ports to those in the log files, and loads the most recent matching usernames into an array. It then does some database magic to match the usernames to customers and outputs the result as a JSON file which can be parsed using libraries available in most programming languages. This script is run by Cron every minute.

Using an AJAX intranet web page that reloads and parses the JSON file every minute, I can provide a list of current users to our technical support staff that’s internal to our company network and does not require direct access to the servers.

[Edit: It should be noted that TShark chews up RAM pretty quickly.  I have it restarting on a daily basis so it doesn't eat all the system memory.  There are probably other tools that can do the job, or you could write a custom parser that's more efficient if that's a concern.  I chose TShark because it was the easiest solution to implement out-of-the-box.]

I haven’t often discussed my work on the Internet.  The main reason is it’s hard to know where to draw the line between proprietary corporate information and public knowledge.  What’s proprietary one minute may be public the next, but it’s not my business to make that call.  That said, work is a major part of my life and to avoid discussing it limits the range of topics I can blog about.  I often get emotional about the things going on at work, be they exciting, boring, or frustrating.  I won’t be able to blog about everything I do at work, but I will try to say what I can.

I work as Network Administrator for Cansel Survey Equipment (Note to spammers:  No, we don’t need your 3rd party IT services, stop calling!).  Though that’s my official title it would be more appropriate to describe my function as “Jack of All Trades.”  In a given day I may be networking, programming, doing desktop support, managing servers, updating spam filters, assisting with GPS base station installs, analyzing data or researching for new projects.  Our IT department deals primarily with the world inside Cansel and very seldom do we support anyone outside our offices.  I greatly enjoy this aspect in that I have gotten to know the names (and often faces) of everyone I support.

Part of my job is assisting in the administration of Can-Net, a cross-Canada VRS network.  Among other things, this involves the occasional configuration and installation of new servers.  Typically we’ve been using pre-built big name systems but lately our network has expanded to the point where we really need to ramp up our storage capacity, data access speed, and processing on one server in particular.  So, we’re putting one together ourselves.  This brings me to my latest point of glee:  The Storage Beast.

This system (when complete) will hold up to 24 SATA or SAS hard drives.  It has dual Quad-Core Xeon processors.  It is awesome and gives me warm fuzzies inside.  Tomorrow I will (among other things) be continuing the setup process and hopefully get the system set up to a point where I can really see what it’s capable of.

Another part of my work on Can-Net has been the programming of an internal interface between our GPS Infrastructure software backend and our company ERP software backend.  My work here has primarily been in Ruby and Ruby on Rails.  I don’t think I can offer much more detail on that at the moment, but suffice it to say I’ve been greatly enjoying the project.  Coding in Ruby has helped me rediscover a love of programming that I had lost sometime after becoming frustrated with PHP.