DSP-10 File Formats
And some information on interpreting the file data...
The DSP-10 provides for saving of various data to files. This may be
useful for later data analysis, for recording an important event, or
for diagnosing a problem. This section collects the various formats
together for reference. Descriptions of the data items are included,
but additional descriptive material may sometimes be available in the
source code.
All DSP-10 output files are straight ASCII text. The entries are
multiple "data records," each consisting of a single line, terminated
by a carriage return and a line feed. In the file descriptions, the
carriage return and line feed are referred to as <CR> and
<LF> where the angled brackets indicate that these are individual
bytes, and not the character strings, "CR" and "LF". No end of file
mark is used. In almost every case, data items within a record are
separated by spaces, usually one. These files can be opened in
programs, such as Excel, by specifying "Delimited" with "Spaces" as
delimiters.
The following formats are covered here:
Full Spectral Data (ALT-F)
LTI Spectral Data
EME-2 Spectral Data
Clock Diagnostic Data File
Screen saves, in GIF format, are covered
separately.
Format info Revised 21 Aug 2006
Full Spectral Data - This data file is controlled by the ALT-F
(or -f) key command. The file name is always UHFA_1.DAT and the path is
specified by the configuration variable, fdata_path, unless the path
specifies a floppy drive (A: or B:). If no path is specified, or the
path is to a floppy drive, the file will be in the same directory as
UHFA.EXE. This file contains the entire waterfall data in raw form, and
so becomes large very quickly.
The powers are normally treated as relative values, in the sense that
the noise can be determined from frequencies without signals, and the
signal levels then deduced by there increase in level over the noise.
The format differs between EME-2 and all other modes, and are covered
separately here. For all modes except EME-2, the format consists
of alternating HA and HB data records, with HA first, as follows:
HA the record type
year as 2006
month 1 to 12
day 1 to 31
hour 0 to 23, UTC
minute 0 to 59
second 0 to 59
<CR><LF>
HB the record type
fft_bw
rfgain
noncohave SpecAve setting on screen
do_window 0=None 1=Tukey25, 2=Hamming, 3=BH92
dsp_deltap
mode+bits*
0 0 0 16 future data items
<CR><LF>
pwrdat**
<CR><LF>
* mode+bits is a single number representing the
current mode as 4* mode+16*bits_8_16. The mode is 0 to 7 and the
bits_ 8_16 is either 0 or 1. This data item was historically useful,
but with growth in number of modes, has become ambiguous. Do not use
and it will be replaced by one of the "future data items," eventually.
** pwrdat consists of the the lowest frequency 256 spectral powers.
That is, the left half of the screen for modes like USB, and all of the
screen for LTI. The powers are not in dB, but are in x.xxxxEyyy
"scientific format," where x.xxxx is a number from 1 to 10 and yyy is a
power of 10 as a multiplier. These powers are arranged as 32 lines of 8
each, with a CR and LF at the end of each line. The powers are the
integrated levels for each spectral bin taken over the time set by
noncohave (SpecAve on the screen).
LTI Spectral Data - This file type can be in
operation at the same time as the Full Spectral Data type, listed above .
This data file is available only in LTI mode, with Beacon Overlay in
place. Writing is controlled by the "Save LT Data to Disk" check box in
the SCRL-F2 Beacon dialog box. The file name is ULTmdd_y.DAT. The
leading "U" can be changed by the configuration file variable
file_ident. The value for m is in hexadecimal, with 1 for January, 9
for September, A for October and C for December. The day, dd is 0
to 31 and the year, y, is the last digit of the year. The path is
specified by the configuration variable, fdata_path. If no path is
specified, the file will be in the same directory as UHFA.EXE.
The data being saved corresponds to the "yellow trace" in LTI. This is
affected by the "Clear LT Data after Save" check box in the SCRL-F2
Beacon Dialog box. If this box is not checked, the averaging continues
until manually cleared (CTRL-W). The "count" value, below, and the
averaging of the data will continue to reflect this setting. The powers
are expressed in dB, and normally are treated as relative values, in
the sense that the noise can be determined from frequencies without
signals, and the signal levels then deduced by their increase in level
over the noise.
The limits on the .DAT files are: 5760 lines (every 15 seconds,
for 24 hours) 241 bins (+/-120 plus the center bin) This bin
limitation corresponds to a DSP-10 configuration file entry:
lt_b_save_bins 120
the 241 bin limit is:
565 Hz for a SpecAnl width of 1200 Hz.
1130 Hz for a SpecAnl width of 2400 Hz
2259 Hz for a SpecAnl width of 4800 Hz
The record type is HL, entries are separated by a space and records end
with CR LF. A typical record is:
HL 2006 7 19 0035 0 2 0 300 107 355 10368.032000 290 ***DATA*** CR LF
where
HL the record type
year as 2006
month 1 to 12
day 1 to 31
hour minute 0 to 2359, UTC Time
second 0 to 59 Seconds
fft_bw * DFT noise bandwidth, Hz SpecWidth, 0 to 2
do_window** window type, 0 to 3
b_receive_sec Beacon receive time in seconds
lt_i3 Index Number of saved center bin
count Number of DFT results averaged and included in this record
freq Center frequency of data, in Hz, including transverters Freq
noise_temp Configuration file variable, called eme2_te, Kelvins Te
Data ***
<CR><LF>
* 1200 = 0, 2400 = 1, 4800 = 2
** None = 0, Tukey25 = 1, Hamming = 2, BH92 Window = 3
*** The data entries include an equal number of bins (lt_b_save),
above and below the center frequency bin (lt_i3) so there are always an
ODD number of entries. They start from lowest frequency and go to
highest.
Two experimental programs are available for interpreting the LTI data
from a DAT file. Called ULT.EXE and ULTS.EXE, they are available to
those with an active need. They lack the documentation for general use.
Contact W7PUA.
SUM FILE - This format corresponds to the output of the
processing program ULT.EXE. The LTI DAT file, described above, is
analyzed record-by-record to extract various parameters from the
data. Several parameters, such as the average noise power is
computed in ULT for the entire LTI frequency range covered by the DAT
file. Other parameters, such as peak signal level, is computed
for a variable number of freuency bands, within the total range.
The number of bands ranges from 1 to 5, under the users control.
The definitions of the output quantities is shown below.
The SUM file definition changed with the addition of multiple band
capability to ULT.EXE; the definitions here are for ULT.EXE, version
1.93 or later.
The ULTmdd_y.DAT file has a line for each data save and it contains the
dB levels (relative) for all the bins being saved (as selected in the
ALT-B box). For each line in the ULTmdd_y.DAT file, a line is
created in the .SUM (summary) file. The .SUM file file inherits
the name from the ULTmdd_y.DAT file and will have the .SUM suffix. The
data items in a .SUM file line are in ASCII and separated by spaces
(20H). They are are as follows:
cur_rec integer The current record number. Records may be missing.
hhmm integer The time of the data in hour and minute format, UT.
sec integer The seconds into the minute
year integer e.g., 2006
mdhms float month, day, hour, minute and second in unit of days*
pts integer Number of FFT powers averaged to produce .DAT line
freq float Frequency in MHz, to the Hz
resbw float Spacing between bins in Hz
noisebw float Noise bandwidth of bins (includes window effects)
avenoise float The dB average of the 10 noise measurement bins**
basenoise float Receiver noise per DFT bin in dBm
n_bands integer The number of bands in use, 1 to 5
The following items will be repeated (with different numbers)
for the n_bands being used
band_nm string Name of band being analyzed
band_low ineger Lowest bin in the band***
band_high integer Highest bin in the band
peakbin integer Bin where the signal+noise was the strongest
peakdb float dB level (DAT file units) at peak
pkfreq float Frequency of peak, Hz
minbin integer Bin where the signal+noise was the weakest
mindb float dB level at minimum
minfreq float Frequency of minimum, Hz
limlow integer Lowest bin where signal is over noise
limhigh integer Highest bin where signal is over noise
secpkbin integer Second highest peak bin number
secpkdb float Second highest peak dB level
secpkfr float Second highest peak frequency Hz
seclimlow integer Lowest bin where 2nd highest signal is over noise
seclimhi integer Highest bin where 2nd highest signal is over noise
maxsn float S/N of peakdb, in dB
cgfreq float Power weighted center frequency (CG), Hz
totpwr float Total power in limlow to limhigh band, dBm
pkpwr float Power in peakbin, dBm
3dbbw float 3 dB bandwidth of peak signal, Hz
6dbbw float 6 dB bandwidth of peak signal, Hz
10dbbw float 10 dB bandwidth of peak signal, Hz
<CR><LF>
* Often it is desireable to plot data according
to some linear time scale. The quantity mdhms makes this easy as it
combines the oddball-based time units into the day of the year and a
fraction of a day. Thus each day is 0 to 1 and noon UTC of
January 3 would have mdhms=2.5000 as it is halfway to between the
2nd and 3rd days of the year. As another example, February 2nd at
1:12 and 30 seconds AM UTC would be
mdhms=31+1+(1/24)+(12/1440)+(30/86400) or 32.0503472.
** This noise is in relative units, with the
same reference as the data in the DAT file.
*** Bands are specified in terms of frequencies
in ULT. To be consistent with SUM file outputs, they are specified here
in terms of DFT bins. To convert to the equivalent bin center
frequency, multiply the bin number by resbw.
Comments on SUM Files - maxsn is a good measure of
signal strength. If it is over about 8 dB, it is very likely a
signal, not noise. One can key on this to ignore noise.
cgfreq is probably the best measure of the "average" Doppler. Either
6dbbw or 10dbbw might be a measure of the total Doppler shift during
the data period.
By taking short data periods, like 15 , or 30 seconds, the Doppler rate
can be approximated.
An example line (record) of a SUM file that uses 2-bands (this is all
on a single line in the file, and broken here for readability):
1 0000 00 2006 199.0000000 466 10368.032200 9.37500 9.37500 54.50 -167.14 2
10_GHz 64 107 97 54.64 909.4 77 54.31 721.9 39 41 94 54.57 881.2 94 94 -14.97 909.0 0.04
-182.12 5.5 10.9 44.1
24_GHz 107 151 142 56.04 1331.2 132 54.25 1237.5 77 93 151 54.59 1415.6 151 94 -3.72
1333.3 3.49 -170.87 80.9 113.1 132.0 <CR><LF>
SM1 FILE - An additional output from ULT.EXE is the SM1 file.
This is sometimes useful for studying signal levels and bandwidths, but
it is restricted in content (a subset of the SUM file):
yrtime float Decimal date and time of data, 0 to 365 (or 366)
-------- And as in SUM, the following are repeated n_bands times ----------
totpwr float Total power in limlow to limhigh band, dBm
cgfreq float Power weighted center frequency, Hz.
6dbbw float 6 dB bandwidth of the peak signal.
<CR><LF>
An example of a line (record) from a SM1 file:
1 199.0000 -181.64 909.0 10.9 -161.71 1333.3 113.1
EME-2 Spectral Data - As for the Full Spectral Data, this data
file is controlled by the ALT-F (or -f) key command. The file name is
always UHFA_1.DAT and the path is specified by the configuration
variable, fdata_path, unless the path specifies a floppy drive (A: or
B:). If no path is specified, or the path is to a floppy drive, the
file will be in the same directory as UHFA.EXE.
The powers are normally treated as relative values, in the sense that
the noise can be determined from frequencies without signals, and the
signal levels then deduced by there increase in level over the noise.
The format consists of alternating HA and HE data records, with
HA first, as follows:
HA the record type
year as 2006
month 1 to 12
day 1 to 31
hour, min 0 to 2359, UTC
second 0 to 59
<CR><LF>
HE the record type
rfgain rf gain, 0 is 100, 1 is 94, etc.
do_window window type 0=None 1=Tukey25, 2=Hamming, 3=BH92
fft_bw 0=1200 Hz, 1=2400, 2=4800
paveave Running ave of ave power* p.ppppppEpp
pave bin 30 to 255 ave power in q.qqqqqqEqq format
0 0 For future use
pd16 Spectral data** x.xxxxEyyy
<CR><LF>
*pave is the average power of the received data.,
over a relatively large bandwidth of DFT bins 30 to 255. This is source
of the dB number in the bottom line, right corner for most lines.
paveave is a running average of the pave values, where the decay value
is 0.95. These items can be used to exclude noise bursts by comparing
the two and omitting data when pave exceeds paveave by some
ratio. This is done in the EME-2 operation, and the ratio is set in the
ALT-B box as "NB Ratio."
** pd16 consists 21 data points of spectral powers, corresponding to
the 10 DFT bins below the center frequency, the center frequency, and
10 points above. The powers are not in dB, but are in
x.xxxxxxEyyy "scientific format," where x.xxxx is a number from 1 to 10
and yyy is a power of 10 as a multiplier. The powers are the integrated
levels for each 2-second reception period.
Clock Logging File - UTIME.DAT - Time File UTIME.DAT is for
diagnostic trouble shooting. It records to a file, in the same
directory as UHFA.EXE, a summary of operations such as the 1 minute
updates, or GPS sets. Each record is a line of ASCII data, with space
separated entries as follows (some entries are zero if no GPS is used):
TM starts all lines
record type: 0=minute updates, 2=GPS set, 4=PC clock correction
day
month
year
hour
minute
second
hundredths of a second
seconds since 1 Jan 1970
dt clock error
seconds since 1970 for last midnight GMT
clock_speed
total millisec delay for last GPS set
gps data valid (LSB)
number of satellites at last GPS set
s/n of top 3 satellites at last GPS set
gps receiver status (0 if not UT+)
gps settings (see below)
text record type and settings again in Hex
<CR><LF>
If the record type is 2 (GPS time set), the base
set, shown above, is followed by
sec.hund of PC clock when GPS time came from GGA message
sec.hund at DCD 1PPS pulse
sec.hund after GPS set to sec.00 PC
milliseconds of delay from gps_msdelay
milliseconds before first DCD transition (can be 0)
milliseconds between DCD 1PPS transitions
If the record type is 4 (PC clock update), the
base set is followed by
seconds before update of PC clock
hundredths of a second before update of PC clock
---- End of record description ----
GPS Settings saved in the file are best seen by
converting to binary or hex, as they are arranged by bits. They are
also in hex at the end of the entry above called "text record type."
The bits show several variables from the configuration file:
Bits 0, 1, 2 gps_type
Bits 3, 4, 5, 6 gps_tdelay
bit 7 gps_dcd_pol
bits 8, 9 gps_sw_gps
bits 10, 11 gps_sw_dsp10
For instance, KD7TS, using a Sandpiper GPS, has
1554, which in binary is 01 10 0 0010 010 or
gps_type=010 (NMEA GPS)
gps _tdelay=0010 (2 sec)
gps_dcd_pol=0 (logic 0 to 1)
gps_sw_gps =10 (2)
gps_sw_dsp10=01 (1)
Table of Contents
V396