fsk_receiver.html

<!DOCTYPE html>
<html>
  <head>
    <meta charset='utf-8'>
    <meta http-equiv="X-UA-Compatible" content="chrome=1">

    <link rel="stylesheet" type="text/css" href="stylesheets/stylesheet.css" media="screen">
    <link rel="stylesheet" type="text/css" href="stylesheets/pygment_trac.css" media="screen">
    <link rel="stylesheet" type="text/css" href="stylesheets/print.css" media="print">

    <title>RF Testing Methodology by NCC Group</title>
  </head>

  <body>

    <header>
      <div class="container">
        <h1>RF Testing Methodology</h1>
        <h2>www.nccgroup.com</h2>

        <section id="downloads">
          <a href="https://github.com/nccgroup/RFTM/zipball/master" class="btn">Download as .zip</a>
          <a href="https://github.com/nccgroup/RFTM/tarball/master" class="btn">Download as .tar.gz</a>
          <a href="https://github.com/nccgroup/RFTM" class="btn btn-github"><span class="icon"></span>View on GitHub</a>
        </section>
      </div>
    </header>

    <div class="container">
      <section id="main_content">


<h2>
<a id="developing-a-digital-fsk-receiver-step-by-step" class="anchor" href="#developing-a-digital-fsk-receiver-step-by-step" aria-hidden="true"><span class="octicon octicon-link"></span></a>Developing an FSK receiver step-by-step</h2>

<h3>
<a id="configure-the-gnu-radio-environment" class="anchor" href="#configure-the-gnu-radio-environment" aria-hidden="true"><span class="octicon octicon-link"></span></a>Configure the GNU Radio environment</h3>

<p>When you first start GNU Radio Companion, you are presented with an Options block and a Variable block. The Options block allows you to enter parameters such as the Title, Author, Description and also allows you to decide if the GUI elements will use the WX or QT environments (WX is the default).</p>

<p><strong>Hint: When using GNU Radio Companion if you cannot find the block you are looking for in the right-hand pane, click on the magnifying glass in the menu bar and this will enable you to do a text search</strong></p>

<p><img src="images/grc_top_block.png" alt=""></p>

<p>The sample rate must be at least twice the maximum frequency (of the modulated data - not the carrier signal), but the higher the sample rate, the clearer the signal will be, which initially will help aid signal identification. The default is 32,000, but I would recommend at least 4,000,000 (4e6 is an easier way to write this, as you don't need to count the zeros!)</p>

<p><img src="images/grc_samp_rate.png" alt=""></p>

<h3>
<a id="configure-the-receiver" class="anchor" href="#configure-the-receiver" aria-hidden="true"><span class="octicon octicon-link"></span></a>Configure the receiver</h3>

<p>Depending on which SDR you have will depend on which options you set here. All of the main receivers can be controlled using the "osmocom Source". The "Device Arguments" settings for different SDRs can be found <a href="http://sdr.osmocom.org/trac/wiki/GrOsmoSDR#Devicespecification">here</a>. However, if you are using an RTL-SDR, you can also just use the "RTL-SDR Source".</p>

<p><img src="images/grc_osmocom_source.png" alt=""></p>

<p>The sample rate will already be pre-populated with the "samp_rate" variable. Set "Ch0: Frequency" to be "centre_freq" and then create a new variable called "centre_freq" with a value near your target carrier frequency.</p>

<p><img src="images/grc_centre_freq.png" alt=""></p>

<p>Although you may need to increase the gains at a later stage if the signal is too weak, initially, use the following settings:</p>

<ul>
<li>  Ch0: RF Gain - 0dB</li>
</ul>


<ul>
<li>  Ch0: IF Gain - 10dB</li>
</ul>


<ul>
<li>  Ch0: BB Gain - 10dB</li>
</ul>

<p>Leave everything else at the defaults.</p>

<p>Select a "WX GUI FFT Sink" and connect it to the output of the receiver by clicking on the receiver output and then clicking on the "WX GUI FFT Sink" input - an arrow will link them together (if you want to unlink two blocks, click on the link and drag - it will disappear).</p>

<p><strong>Note: By default the inputs and outputs are blue on these blocks, this denotes the type (blue = complex data, orange = float etc.) If the colours do not match on the connecters you are trying to connect together you will get a type-mismatch error and the arrow will go red.</strong></p>

<p>In "WX GUI FFT Sink" set "Peak Hold" to "On", this will ensure that any signals that appear will stay displayed, even if they are transient:</p>

<p><img src="images/grc_wx_gui_fft_sink.png" alt=""></p>

<p>You are now ready to start the receiver - click on the gears icon in the menu bar and the flow-graph will start. The FFT sink displays 4MHz of bandwidth (your sample rate is 4 million and therefore the receiver is simultaneously sampling 4 million signals every second). The display is showing you everything that is being received in the frequency domain, so every peak is a different signal. If you were to tune through the FM broadcast band (87.5MHz to 108MHz) each peak would be a different radio broadcast signal. FFT is <a href="http://en.wikipedia.org/wiki/Fast_Fourier_transform">Fast Fourier Transform</a> - feel free to read all about it if you enjoy your maths! But understanding how it works is not required. Below is the output I saw when I started my receiver using the centre frequency 867.3MHz, which is in the <a href="http://en.wikipedia.org/wiki/ISM_band">ISM band</a> that's generally used for devices that form the Internet of Things:</p>

<p><img src="images/grc_received_baseband.png" alt=""></p>

<p>I have no idea what this signal is, but thought it would be a great opportunity to test out my methodology against, so the remainder of this receiver development example is based on this signal. Looking at the frequency, it is around 1Mhz higher than my centre frequency, so about 868.3MHz.</p>

<p>The flow-graph can be stopped by closing the FFT window.</p>

<h3>
<a id="visualising-the-signal" class="anchor" href="#visualising-the-signal" aria-hidden="true"><span class="octicon octicon-link"></span></a>Visualising the signal</h3>

<p>So far we know that there is a signal (which happens to be repeating periodically) at around 868.3MHz, but we have no idea about what that signal is. The next stage is to save it to a file so we can do some post-processing. Add a "File Sink" and connect it to the output of the "osmocom Source". Set the filename to something sensible - I always include the sample rate within the filename, so use something like "4M_received_baseband". Restart the flow-graph by clicking on the gears icon and record enough data so that it includes the signal of interest.</p>

<p><strong>Note: SDRs generate very large amounts of data - a recording lasting only a few seconds can result in many tens of megabytes of data, so experiment first!</strong></p>

<p>Now we have a recording we can visualize the signal a bit better using a tool called <a href="http://www.baudline.com/">Baudline</a>, which is an excellent tool for RF visualization that can be very helpful for signals identification. Unfortunately a bug in Baudline means that it does not like file sizes greater than 50Mb. So, before you can load it into Baudline you'll need to split your file into 50Mb chunks:</p>

<p>$ split -b50m 4M_received_baseband part</p>

<p>This should result in a series of files called "parta, partb, partc etc."</p>

<p>Start up Baudline:</p>

<p><img src="images/baudline_default.png" alt=""></p>

<p>Right-click to open the main menu -&gt; input -&gt; open file. Make sure you select your "partx" file rather than "4M_received_baseband".</p>

<p>A window will appear with a range of options; select the following:</p>

<ul>
<li>  Sample rate: "Custom - 4000000"</li>
</ul>


<ul>
<li>  Channels: 2</li>
</ul>


<ul>
<li>  Tick "quadrature" and "flip complex"</li>
</ul>


<ul>
<li>  Decode Format: 32 bit float</li>
</ul>

<p><img src="images/baudline_load_file.png" alt=""></p>

<p>Then click "Open"</p>

<p>The file will be displayed graphically with time vertically and frequencies displayed horizontally.</p>

<p><img src="images/baudline_signal.png" alt=""></p>

<p>You may need to play with the colours to make the signal clearer (main menu -&gt; input -&gt; colour aperture)</p>

<p>If you look carefully at the output, you can see a squiggly yellow line - that is the signal! You can zoom in and out of the time domain using Alt-up arrow and Alt-down arrow. If you want more detail in the frequency domain you need to change the FFT "bin size" (main menu -&gt; process -&gt; transform size).</p>

<h3>
<a id="signal-identification" class="anchor" href="#signal-identification" aria-hidden="true"><span class="octicon octicon-link"></span></a>Signal identification</h3>

<p>If you zoom in it looks like this:</p>

<p><img src="images/baudline_signal_zoomed.png" alt=""></p>

<p>As the line is moving from left to right and the horizontal axis is frequency, this looks very much like Frequency Shift Keying. Also, when data is sent over RF it always starts with something called a "preamble", which is used to help denote the start of some data and also synchronise the receiver with the data. Preambles are normally 101010101010...</p>

<h3>
<a id="centering-the-signal" class="anchor" href="#centering-the-signal" aria-hidden="true"><span class="octicon octicon-link"></span></a>Centering the signal</h3>

<p>Going back to GNU Radio companion, we now need do some more processing to make sure the signal is centered. First, replace the receiver with a "File Source" block that will replay the "partx" file that contains the signal. Also, set the file repeat to "Yes". Next, add a "Throttle" block after the "File Source", so that we don't overwhelm the CPU on the PC running GNU Radio Companion.</p>

<p>Add a "Signal Source" block with the Frequency set to "-950e3" (I said the signal looked like it was about 1MHz higher than the centre - it was actually 950KHz)</p>

<p><img src="images/grc_signal_source.png" alt=""></p>

<p>Add a "Multiply" block and connect the "Throttle" output and "Signal Source" output to the two "Multiply" inputs. Add the "Multiply" output to the "WX GUI FFT Sink" and your flow-graph should look like this:</p>

<p><img src="images/grc_multiply.png" alt=""></p>

<p>Start the flow-graph and the signal will now be centred:</p>

<p><img src="images/grc_fft_centred.png" alt=""></p>

<h3>
<a id="filtering" class="anchor" href="#filtering" aria-hidden="true"><span class="octicon octicon-link"></span></a>Filtering</h3>

<p>We now want to focus in on the signal of interest and filter out any other signals present in the baseband we have recorded. Add a "Low Pass Filter" block between the "Multiply" block output and the "WX GUI FFT Sink" input.</p>

<ul>
<li>  Set "Cutoff Freq" to 150e3 - this is the maximum frequency (of the modulated signal) that we want to display</li>
</ul>


<ul>
<li>  Set "Transition Width" to 50e3 - this is the bandwidth over which the filter will act i.e. the smaller the number the more aggressive the filter.</li>
</ul>

<p>Restart the flow-graph and you should see something like this:</p>

<p><img src="images/grc_filtered_signal.png" alt=""></p>

<h3>
<a id="demodulating" class="anchor" href="#demodulating" aria-hidden="true"><span class="octicon octicon-link"></span></a>Demodulating</h3>

<p>We are making the assumption that based on the output we observed in Baudline, the modulation scheme is FSK (to be precise, "2-FSK" or "Binary FSK" where the frequency is shifted one way for a "0" and the other for a "1"). Therefore, we need to add an FSK demodulator - there are many different modulators available within GNU Radio Companion, but the one we want is the "Quadrature Demod".</p>

<p>Add a "Quadrature Demod" and set the "Gain" to be 1.</p>

<p>Add a "File Sink" and set its input type to be "float" (orange coloured) and set the filename to be something like "4M_filtered_demod"</p>

<p>Connect the output of the "Low Pass Filter" to the input of the "Quadrature Demod"</p>

<p>Connect the output of the "Quadrature Demod" to the input of the "File Sink"</p>

<p>In "File Source" set "Repeat to "No"</p>

<p>Your flow-graph should now look like this:</p>

<p><img src="images/grc_demod_save.png" alt=""></p>

<p>Run the flow-graph and it will produce a file that hopefully will contain demodulated data.</p>

<h3>
<a id="visualising-the-data" class="anchor" href="#visualising-the-data" aria-hidden="true"><span class="octicon octicon-link"></span></a>Visualising the data</h3>

<p>We should now have a demodulated file that can be loaded into Baudline.</p>

<p><strong>Note: Remember, if your "4M_filtered_demod" file is larger than 50Mb then you will need to split it into chunks again.</strong></p>

<p>Start Baudline and load your demod file - use all the same parameters as before, but this time change "Channels" to 1 (it is no longer a complex baseband signal - it is a demodulated signal). Click "Open" and it should look something like this:</p>

<p><img src="images/baudline_demod1.png" alt=""></p>

<p>Ensure that the separate "Waveform" window is open (main menu -&gt; displays -&gt; waveform)</p>

<p>The output in the "Waveform" window can be zoomed in or out vertically and horizontally using Alt + cursor keys. To move the signal up and down just use the cursor keys. The output should initially look something like this:</p>

<p><img src="images/baudline_waveform1.png" alt=""></p>

<p>Although it seems quite clear in the main Baudline window, you'll need to search for your demodulated signal :-)</p>

<p>By scrolling out horizontally, you should see lots of noise and then a section of no noise - this is your signal:</p>

<p><img src="images/baudline_waveform2.png" alt=""></p>

<p>Unfortunately, when you try and zoom in, the data is almost unrecognizable, because of the level of noise affecting it. If this is the case it means that you need to reduce the "Cutoff Freq" and "Transition Width" of your "Low Pass Filter". After some experimentation, some reasonable values were chosen:</p>

<ul>
<li>  "Cutoff Freq" = 50e3</li>
</ul>


<ul>
<li>  "Transition Width" = 20e3</li>
</ul>

<p>Now when the signal was loaded in Baudline it looked like this:</p>

<p><img src="images/baudline_demod_cleaner.png" alt=""></p>

<p>These are the data bits represented by the original "squiggly line" we saw in Baudline when we looked at the complex baseband signal. It also confirms that the data was modulated using 2-FSK.</p>

<h3>
<a id="further-filtering-and-clock-recovery" class="anchor" href="#further-filtering-and-clock-recovery" aria-hidden="true"><span class="octicon octicon-link"></span></a>Further filtering and clock recovery</h3>

<p>As we can see from the signal, it still looks very noisy, so we need to do some further, more accurate filtering and in order to do this we need to estimate the Baud rate (the number of symbols per second), but first we need to discuss how the data is represented. It looks like the peak represents a "1" and a trough for a "0", however it's not quite as simple as that; it is in fact encoded using something called "Manchester encoding". Manchester encoding is what is known as a <a href="http://en.wikipedia.org/wiki/Line_code">"line code"</a> and ensures frequent voltage transitions, directly proportional to the clock rate, which helps clock recovery. What this actually means is that a "0" is represented by a transition to low and a "1" is represented by a transition to high.</p>

<p><img src="images/manchester_encoding.png" alt=""></p>

<p>If we look at the signal with respect to the scale, we can see that by choosing a peak, its just short of 2.5 bits (transition down, transition up, start of transition down) per millisecond, which if you know your standard Baud rates (they don't always conform to standard Baud rates!), it looks like 2400Baud (simple FSK data rates are usually very slow, as data packets are typically quite small)</p>

<p><img src="images/baudline_demod_cleaner_rate.png" alt=""></p>

<p>Ok, now we think we have the Baud rate, we can accurately add a new "Low Pass Filter" in between the "Quadrature Demod" and the "File Sink". The settings should be as follows:</p>

<ul>
<li>  Cutoff Freq = 2400 (the Baud rate)</li>
</ul>


<ul>
<li>  Transition Width = 1200 (half the Baud rate)</li>
</ul>

<p>Also, you may have noticed that the flow-graph has been running very slowly since we started demodulating the data. Let's try and work out why:</p>

<p>The sample rate is currently 4,000,000 and the Baud rate is 2400, so if we divide 4,000,000 by 2400 we get the number of samples per symbol - 1666! That's why it's running slowly... we don't need anywhere near 1666 samples to represent each bit.</p>

<p>In the "Low Pass Filter" set "FIR Type" to "Float-Float (Decimating)" and set "Decimation" to 100. This will decimate the signal (reduce the sample rate) by a factor of 100</p>

<p>Now re-run the flow-graph and load the resulting demodulated file into Baudline and the "Waveform view" should now look like this:</p>

<p><strong>Note: When you load the file back into Baudline don't forget that the sample rate has been decimated by 100 and is now, therefore, 40,000 instead of 4,000,000</strong></p>

<p><img src="images/baudline_demod_really_clean.png" alt=""></p>

<p>We're not quite ready to recover the bits yet though, as the signal should be alternating about 0dBm (the centre line in the waveform view. It's actually up towards +90dBm, so it needs to be reduced.</p>

<p>Add a "Add Const" block in between the second "Low Pass Filter" and the "File Sink" and set the "Constant" value to be -0.055 (this value was determined through trial-and-error), to reduce the signal level down. The output should now look like this:</p>

<p><img src="images/baudline_final_signal.png" alt=""></p>

<h3>
<a id="data-recovery" class="anchor" href="#data-recovery" aria-hidden="true"><span class="octicon octicon-link"></span></a>Data recovery</h3>

<p>Now we are ready to recover the digital data. First we need to perform clock recovery - add a "Clock Recovery MM" block and set the following values:</p>

<ul>
<li>  "Gain Omega": 0.01</li>
</ul>


<ul>
<li>  "Mu": 0</li>
</ul>


<ul>
<li>  "Gain Mu": 0.1</li>
</ul>


<ul>
<li>  "Omega Relative Limit": 0.01</li>
</ul>

<p>If you're interested in what these values mean then read "Timing Recovery in Digital Synchronous Data Receivers" by Mueller and Muller.</p>

<p>The final value we need is "Omega" - this is the number of samples per symbol. We previously calculated this to be 1666 and so decimated the signal by a factor of 100 so "Omega" now needs to be set to 16.66 (1666 / 100).</p>

<p>Add a "Binary Slicer" to the output of the "Clock Recovery MM" block</p>

<p>Add a "File Sink" to the output of the "Binary Slicer" - you'll need to set the input type of the "File Sink" to "Byte" (pink coloured). Set the filename to be something like "bitfile"</p>

<p>Your final flow-graph should look something like this:</p>

<p><img src="images/fsk_full_flowgraph.png" alt=""></p>

<p>Open up the "bitfile" file in a hex editor. Remember the preamble of "0101010101..."? If you count them, there are four bytes worth (32bits) so you need to search for the preamble in the "bitfile", but if you look carefully, you can see that each byte in the file represents one bit i.e a "0" bit is represented by the byte "00" and a "1" bit is represented by the byte "01", so to search for the preamble you need to search for: "0001000100010001000100010001000100010001000100010001000100010001". The result of the search is shown below, the rest of the data follows the preamble:</p>

<p><img src="images/hex_editor_bits.png" alt=""></p>

<p>You have now successfully retrieved the data from an RF transmission!</p>

<p>I still have no idea what this signal actually originates from - maybe one of my neighbours has an automatic garage door, or it could be a wireless central heating controller or some other IoT device.</p>

<h3>
<a id="communicating-with-the-receiver-in-real-time" class="anchor" href="#communicating-with-the-receiver-in-real-time" aria-hidden="true"><span class="octicon octicon-link"></span></a>Communicating with the receiver in real-time</h3>

<p>Rather than using a file as a sink you can use a "TCP Sink" block, bind it to a socket and receive data from it via TCP</p>


   </section>
    </div>

    
  </body>
</html>