Show by Label

Friday, October 16, 2020

Monitoring, measuring & logging a GPSDO

This is actually a continuation of another project, a high precision 10MHz GPS disciplined oscillator (GPSDO), also described on my blog here

Everything that I discuss here will eventually be posted in a dedicated Github project, available here.

This is very much a work in progress, so keep in mind that things will change, so please come back often to keep track of the developments.

Monitoring, measuring and logging

During the development of my GPSDO's, I actually built three complete units, I wanted to monitor, measure and log the results so I could analyze the data later, and in different ways. Most of the changes take place over a long period of time, mostly several days, so it's easier to look at changes when you can graph the data. I have been doing that for many months now, but I wanted more.

My additional design goals

1. I wanted to move away from the RPi's currently sitting on top of my GPSDO's, and being fed by an additional 5V wall-wart supply. 

2. When I wanted to look at the NEO performance with u-center, or reprogram them, I needed to open-up the enclosure and add a serial to USB connector and cable. Or leave that cable connected through a slit in the enclosure. This is cumbersome, so I also wanted the RPi to have access to the NEO serial data, as a minimum just listening, but with the option to program it through the RPi as well, although this is way more complicated to realize. 

3. What is also cumbersome at the moment, is that I have to open-up the enclosure to de-solder/solder the 10uF capacitor on the reset pin of the Arduino, to avoid a general reset when you plug the USB cable in, just to reprogram the firmware. Even if I added a jumper, which I should have done, I don't want to open up the enclosure. It really disturbs the performance of the GPSDO. A simple on-off switch on the front- or back-panel could do that job.

4. I found a high precision counter project for the 10MHz, and I wanted to integrated that into the GPSDO.

5. With the additional information of the 10MHz counter and the NMEA data, I also wanted to add a very small OLED display to the front-panel, so I can show the 10MHz count, the number of satellites in view and maybe the DAC or temperatures, whatever...

6.  I'm also considering adding two more SMA connectors for the 1PPS and 10MHz outputs, but then isolated with a transformer. I have noticed losses of lock when I fiddled with the BNC cables connected to the current outputs. These outputs are DC coupled to the circuitry, and the whole GPSDO is floating from earth ground. To counter that from happening at the moment, I always have one BNC cable between the GPSDO and the external trigger input from my scope, which is earth grounded.

To put everything inside the GPSDO enclosure, I needed a smaller foot print for the RPi itself, so I choose to use the model Zero-W. The "W"stands for wireless, which is what I needed to get access to the internet and add email functionality. I also wanted to power the Zero with the already available 5V supply inside the GPSDO. 

To put everything inside the enclosure, I'm designing a small additional PCB that will house the counter hardware, a connector to fit the RPi Zero and the isolation circuitry for the additional 1PPS and 10MHz outputs. 

Bit-banging my way out of a predicament

Because I wanted to monitor the Lars' report, the NMEA messages from the NEO, and Yannick's 10MHz counter, I needed two extra serial ports, something the RPi family does not have. Luckily, there is the equivalent of a SoftSerial that the Arduino has, by bit-banging the GPIO pins of the RPi. That library (pigpio) took me quite some time to figure out, because the bit-banging function does not support a "readline" equivalent. It actually reads the serial data in bytearray chunks, so you need to piece together these chunks in order to make a complete sentence. 

This was rather easy to do for the counter, because it sends out the serial data in a fixed length format. However, things changed dramatically when I tried to collect the NMEA data from the NEO. After some additional programming and testing, I also managed to figure that out. Next up is using the bit-banging software for the Lars' report, which should be a lot easier now that I know how to work with the bit-banging. After that I will start the process of combining it all, and will then show the results soon.
When everything is programmed and working, I will make the decision which of the serial inputs will be going to the "real" UART. This will probably be decided on the processor load, but more importantly, the severity of possible errors, which currently would vote for the counter report.
Below I will describe the various project parts.

Monitoring the GPSDO status report

Since the beginning, I used a Raspberry Pi Model 2B (the "classic" first model, because I have several of them) to capture the output of the status report from the firmware of the GPSDO and add a timestamp in front of every line. After capturing that data into a file, an email program sends that just before the end of a day to my PC for further processing in Excel.

I used two separate Python scripts to do that. They are on the Github site.

Because I will most likely need to move this serial monitoring away from the "real" UART and use bit-banging, the serial monitoring code needs to undergo some serious treatment. I have not yet started that.

Measuring the accuracy of the GPSDO 10MHz output

To really measure the accuracy of the 10Mhz, you need a counter with a precision that can show at least 5-6 decimal digits of the 10MHz frequency. A normal counter will need a gate time that is measured in weeks, so the only viable way to measure this frequency is to use a good reciprocal counter. These instruments are normally way beyond the pocket-money most DIY hobbyists can spare. I know I can't.
So the only cheap method available to me is to use a counter that can measure down to 3-4 decimal digits.
While I was looking for counters that can be used on GPSDO's, I came across an interesting GPSDO project, that uses an integrated  counter to measure the output. 

I found this so intriguing that I ordered the parts to build this counter. Unfortunately, the maximum gate was "only" 1,000 seconds, giving you a mere 3 digits or 1pps. I wanted more, so I contacted the author, Yannick Turcotte, and described my problem. He was very quick in providing me with a test version that could go as high as 10,000 seconds, giving us another digit of precision.
Here is my first attempt with what we called the 10K version. I feed it with the 1PPS of my Bliley GPSDO, and also use the 10MHz output to serve as the clock for the Arduino. The connector you see on top is part of the programming interface.

Even with a 10,000 second gate time, the resolution is "only" 100microHerz. ( 10,000s * 10MHz = 10,000,000.0000 +/- .0001) My Bliley is better as you can see in the TimeLab frequency difference view (note that this is with a TC of 500 which is not the optimal setting):

Timelab reports a frequency of 9,999,999.999 991 948 at 10,000 seconds, so the Bliley is accurate to 5 decimal digits, one more than the counter can measure. If you realize that 1,000 seconds is 2.77 hours, it's clear that another digit of resolution would mean 100,000 seconds or almost 28 hours. Not very practical. The solution of course is to use a good reciprocal counter, but as I mentioned earlier, that is way outside my amount of pocket money, and even more so beyond my skills to build one. 

The caveat of using this counter technique with such a long gate time is that you only measure the average frequency during this gate period. However, in this particular case, with this application, this is not such a big deal, because that's what a GPSDO is supposed to provide anyway. Long term stability and precision.

In any case, just displaying this result, that is only refreshed every 2.7 hrs, is nice but not very useful. Especially when your GPSDO has another digit of precision higher. However, if you can log the results and graph them, you can see the drift over time, or the GPSDO getting "out of specification". Maybe this is just a gadget, I'm going to add it anyway and see what benefits or insights it will bring.

After a lot of testing and fiddling from my side, Yannick has now produced a counter version especially made for this project, that has a remotely programmable gate time of 1,000 seconds to measure with 3 decimal digits, or 10,000 seconds to get 4 decimal digits. It also sends the results to the serial output of the Arduino so it can be captured and manipulated by the RPi. 
The source for this special version is available here :


Raspberry Pi serial caveat

Most of the time I spent testing the counter was by getting a reliable serial connection between the Arduino and the RPi Model Zero-W. This was by using the standard TxD and RxD pins on the connector. After days of struggling, I found that the occasional dropped data and garbage I received was caused by a not-so-well-known change in the firmware for the Model 3, 4 and the Zero-W I'm using. 
The Raspberry Pi models only have one "real" UART. The Raspberry Pi Foundation made a decision a while ago to replaced the reliable UART for what they call a "mini-UART", to provide Bluetooth support using the "real"one, but neglected to communicate this well enough. Note that the Model 2B that I was already using to monitor the GPSDO's, did not have this swap, so when I moved the applications to the Zero-W, I was totally unaware of this change and fell into this nasty trap. The justification from the Foundation for this change was that the serial UART was mainly for console use and therefore less critical. Yeah, right.
So using the "mini-UART" turned out to occasionally produce unreliable results (dropped data or garbage). This was most likely caused by the non-deterministic nature of Linux, so when it did some heavy housekeeping, the mini UART lost data . When you're waiting for one message every 2.7 hours, you have to capture it reliably, or wait for the next one. It took me several days to hunt down the problem and then find the available solution, that is, if you knew what to look for. Grrr.... By no means the first time the Foundation caused issues like this.
How to switch from the "mini-UART" as the default back to the "real" UART.
First, enable the serial port with sudo raspi-config.
For the RPi 3, 4 and Zero-W, you need to swap the mini-UART with the TL011, the real one.
Edit the boot configuration: 
    sudo nano /boot/config.txt
At the end of the file, add:
Save the file and close the editor.
You now need to disable the modem software:
    sudo systemctl disable hciuart
Reboot the RPi.
This allows you to use the following serial device again : /dev/ttyAMA0

Logging the NMEA data from the GPS module

Because of the limited number of serial UART's avalable on the RPi family, I needed to find a kind of SoftwareSerial that would let me use any of the available GPIO ports of the RPi. There are three libraries available that I know of, but I selected to use the pigpio library from Joan, who has been very active on the RPi front right from the start. He or she, is very skilled, active and helpful, so this library is where I started the adventure with. This library has support for bit-banging of the GPIO pins to act as serial TxD and RxD ports.

As I mentioned earlier, the pigpio bit-banging software reads the serial data in irregular chunks in the form of Python bytearrays, which you need to piece together again. Sounds trivial, but was not so easy to do with the NEO NMEA data.

After some inspiration and a lot of testing, I figured out a way to do this reliably, and the script now reads the data without problems for more than a day. Good enough for me.

The code is on the Github.

If you can find a more reliable or a more clever way of doing this, please chime in and post your solution.


The OLED display

The little 0.9" diagonal display will easily found a place on the front-panel. I already worked out how to program it. The test script is on the Github. 


The hardware circuitry

 The schematics and design is a work in progress, but it shows roughly what I'm planning to do.

This part of the circuit is heavily based on Yannick's version, I just stripped away the non-essential parts. There is only one LED left, this is the counting LED that I will put on the PCB, it's there to see if the counter works and does not need to go on the front-panel.
The controlling button is most likely also not needed, but I'll include the parts on the PCB. The inputs for the counter are the 10MHz and the 1PPS coming from the GPSDO. I also tapped these two signals to provide them as isolated outputs on the front or back-panel. The 5V supply comes from the 5V powering the OCXO of the GPSDO.

 Above is the RPi portion of the circuit. The tiny 0.91" display is an i2c version, so only needs the minimum amount of connections. There is a library available that is simple to use. I have another display, a little bit larger, on order and I will decide later on which one it will be. 
The RPi will read the serial data from the counter, the NEO and the Arduino Nano of the GPSDO. At this moment, only listening. The NEO and the counter produce 3V3 level signals, so I use a simple level convertor or series resistor to keep it safe. At 9,600 Baud, this should be sufficient, otherwise I need to use a MOSFET. Since these TxD lines are all going to RPi inputs, there will be no smoke anyway, however, the TXD signals coming from the RPi, if used for the NEO, will need a level shifter. There is already a USB connector on the front-panel to get access to the Arduino Nano so it does not need a TxD connection.
The RPi can select the 1,000 or 10,000 second gate time for the counter so it's under program control.
I also decided to put the fan driver circuit on this board so you don't need a separate PCB.

Putting it all together

I'm first going to produce a prototype on perf board that will go inside the Trimble enclosure. With PuTTY I can get access to the RPi and work on the programs without disturbing the GPSDO. Once that works to my satisfaction, I will work on the PCB layout and also work out where and how I'm going to place it inside the enclosure, and how and where to mount the OLED screen.  

Still more to come... 

Github site with all files is here:

No comments:

Post a Comment