This afternoon I put the last block into Python QAM "modem" version 3: Automatic Gain Control (AGC). The sources can be found at
http://epx.com.br/wav, the m3*.py files, as well as gray.py as a dependency. The feature can be tested by e.g. changing volume of qam.wav file using Audacity.
AGC allows the modem to adapt to different power levels. Up to now, V3 modem was not adapting to the "volume" in WAV file; it assumed that TX always used 90% of dynamic range. V2 already had a crude amplitude measurer which only paid attention to the training header sequence. But a true AGC must adapt to changes mid-flight.
This is easier said than done. The main problem is that AGC must figure out the full dynamic range, that is, the maximum amplitude a symbol may have. But there is no guarantee that the "strongest" symbols will actually appear in QAM signal. And, if they appear, how does AGC "know" that they are the ones?
A crude solution would be to send the training sequence at maximum amplitude every few seconds, which is easily detectable as a no-data sound, but this wastes bandwidth and is not 1337 enough :)
The actual solution that real-world modems employ is to work with power level
averages. It is trivial to calculate the average amplitude of our constellation. Then, we maintain a moving average of QAM signal reception, and compensate for the difference.
V3 uses a one-second moving average, which is short enough to profit from initial training sequence, and adapts fast enough while actual data is being received. TX was changed to send a training sequence at this average power level instead of the maximum possible power level, so AGC does not have to distinguish between training and data signal parts.
Then we have another problem: will actual signal power average match the theoretical constellation average? In theory, this will happen only if all symbols are equally probable. Real-world modems employ a randomizer on bit stream to ensure that.
I was optimistic that AGC would get a good average without a randomizer, but I was wrong. AGC will not work at all without a very random bit stream! It is incredible how ordinary data, like a text file, is so visibly non-random at such a low level.
So, out of necessity, I have implemented a very simple bit randomizer for V3, with a "4th order polynominal". This is a very pedantic way to say that randomizer takes into account the last 4 bits to figure out the next one, which is calculated with simple XOR operations. To be even more pedantic, I even employed a good programming practice this time: a self-test.
# Randomizer: 1 + x**-3 + x **-4
def randomize(b0):
global b1, b2, b3, b4
bs = b0 ^ (b3 ^ b4)
b1, b2, b3, b4 = bs, b1, b2, b3
return bs
def derandomize(b0):
global b1, b2, b3, b4
bt = b0 ^ (b3 ^ b4)
b1, b2, b3, b4 = b0, b1, b2, b3
return bt
rseq = [ int(random.random() * 2) for x in range(0, 25) ]
b1, b2, b3, b4 = (0, 0, 0, 0)
rseqS = [ randomize(b) for b in rseq ]
b1, b2, b3, b4 = (0, 0, 0, 0)
rseqT = [ derandomize(b) for b in rseqS ]
b1, b2, b3, b4 = (0, 0, 0, 0)
if rseqT != rseq:
print "Randomizer is broken"
print rseq
print rseqS
print rseqT
sys.exit(1)
This randomization scheme was copied directly from Fabio Montoro's "Modem e transmissão de dados" book.
It was funny to realize in practice how important the randomizer is, when you simply don't know the power level at RX side and must figure it out based on data stream itself.
Determining power level using averages, even with the help of randomizer, does not have the same precision as knowing it beforehand. This reduces the number of bits per symbol that modem can handle. V3 had reached 16 bits/symbol, now the highest reliable level is 12. (Turning off AGC in RX side makes it go back to 16.)
AGC was the last thing to implement in V3. The next major milestone, V4, will add some form of convolutional coding to increase noise resistance.
