How to record data
From CSL Wiki
Recording acoustic data for future processing is an important starting point for testing and implementing systems using the Acoustic ENSBox.
Contents |
Recording
The easiest way to record audio data is to use the sensors/record_from utility, and save to flash (on /mnt/sd_card) or over nfs. Record from will record the data into a simple "raw" binary file (ends .raw), encoding the samples as signed 16-bit values. If you are recording multiple channels, they will be interleaved. record_from also saves synchronization data in a '.sync' file, which is a text file that maps specific sample indices to local and global time. For this to work, make sure you have "global time" broadcasting set up correctly.
The record_from usage provides this help:
[girod@breeze emstar]$ obj.i686-linux/sensors/record_from -h
Must supply a sensor name and destination path!!
Usage: obj.i686-linux/sensors/record_from (Mixed_Sources)
[-h] [-o <loglevel>] [-v] [--startdir <dir>] [--daemon] [-G <sim_group>] [-N <node_id>]
--sensor <sensor-name> [--sensor_clock <other-clock>] [--start_clock <start-clock>]
[--scale <scale>] [--start <sec.usec>] --dur <sec> --path <file>
--help: Print this message
-o <loglevel>: set the default stdout loglevel
-v: report version information
--daemon: startup as a daemon (detach)
--startdir: chdir to <dir> before running; useful with --daemon
--group_id <id>: Overrides the SIM_GROUP environment variable
--node_id <id>: Overrides the node ID in /etc/id or SIM_ID
--stop-on-startup: stop in misc_init() and wait for GDB to attach
--enable-remote: make devices available over the network via FUSDNet
--sensor <sensor-name>: the sensor to record
--sensor_clock <other-clock>: name of a clock to use if not the sensor's clock
--start_clock <clock-name>: name of a clock to interpret start time, and to
save timestamps in (default CPU, often use gps)
--scale <scale>: the multiplier from samples -> ticks (default 1)
--start <time>: the time to start recording, (default 'now')
--dur <sec>: maximum number of seconds to record
--path <string>: path to save files to
--compress: not supported
--max_buf <buf>: max size of buffer
record_from records sensor data to files, with timestamps
To use record_from to record 20 seconds from the 'single' sensor device (which provides ch0 only) and save the output to the sd_card:
record_from --start_clock gps --scale 20 --sensor_clock /dev/sensors/vxp/all --dur 20 --sensor vxp/single --path /mnt/sd_card/mytest
Note that there were a few extra options there. When using the AENSBox interface you will always need to include those. You don't necc. need to know what these mean, but I describe them below:
- --start_clock gps: Causes the sync file to record timestamps in terms of global time (GPS) rather than local CPU time. This is needed if you want to relate recordings from two different nodes.
- --sensor_clock /dev/sensors/vxp/all: This is the name of the clock that represents the sampling process on the AENSBox. Normally the clock is assumed to be the same name as the sensor itself, but AENSBox has two sensor interfaces (single and all) that share the same clock (the one for all). So when using 'single' you need to specify it to use the clock for 'all'.
- --scale 20: This tells record from that each tick of the sensor clock is scaled by 20. This scaling is a hack to allow time conversions from CPU time to resolve to fractions of a single sample. (Note, at 48KHz each sample is about 20 usec, so for 48KHz rate this also makes the clock conversion convert at close to unit rate.)
To use record_from to record 10 seconds from the 'all' sensor device (which provides 4 channels interleaved) and save the output to the sd_card:
record_from --start_clock gps --scale 20 --sensor_clock /dev/sensors/vxp/all --dur 10 --sensor vxp/all --path /mnt/sd_card/mytest
If you want to do simultaneous recordings, you can use rbsh to start record_from on several nodes at once. Assuming that your interface that is on the LAN with your nodes is eth0:1,
girod@wren:/scratch/girod/emstar$ obj.i686-linux/bin/rbsh -b eth0:1 Welcome to the rbsh remote broadcast interactive shell. Enter shell commands, followed by a blank line to send. rbsh 1> record_from --start_clock gps --scale 20 --sensor_clock /dev/sensors/vxp/all --dur 10 --sensor vxp/all --path /mnt/sd_card/mytest
Will start up the recording at approximately the same time on all nodes. If you have synchronization set up properly, you can then use the .sync files to line up the recordings after the fact.
You can also use rbsh to record on all nodes synchronously (i.e. all recordings start at exactly the same time). To do this you must first telnet to one of the nodes. Then, you use rbsh in the non-interactive (command line) mode:
root@slauson # rbsh -U flood record_from --start_clock gps --scale 20 --sensor_clock /dev/sensors/vxp/all --dur 10 --sensor vxp/all --path /mnt/sd_card/mytest --start_time `gettime --clock gps`
This command will execute the backticked part (gettime --clock gps) locally and then send that command containing that timestamp over the network to all the other nodes, via the flooding adaptor. Note, if you executed the same command in interactive mode, it would not work because the backticked part that determines the start time would be executed on every node rather than just once, and so they would still all start at slightly different times.
Keeping Up
Unfortunately, neither the sd_card nor in some cases the network can keep up with full rate (48KHz) audio. This effectively limits the amount of data you can record continuously to about 20-30 seconds maximum. record_from is designed to limit the in-memory buffering and automatically stop when it exceeds a threshold. In fact it's a little more complicated than that because the kernel buffers stuff that's headed for NFS or sd_card, and NFS keeps a copy of pages written until they are acked, so these can take arbitrary amounts of buffering. When your system starts to run out of memory, things go south quite rapidly, and you may experience audio dropouts as your free memory drops. The bottom line: you can't record long traces at 48KHz.
If you can get by with 12KHz, this may be a better solution. The AENSBox can boot into a "recording only mode" that uses 12KHz rate. Acoustic ranging and positioning does not function in this mode, but you could run that first to localize and then reboot into the other mode. (Note: it would not be impossible to switch more gracefully but it's currently not supported.)
Using lower rates will generally mean you have more CPU and memory to devote to signal processing so it's always best to choose the lowest rate that works for you.
Note, the acutal rate is specified in the .run file:
# Acoustic sensor &sensor_vxp(args="--enable-remote -s 12000");
Note: The VXPocket440 supports certain fixed rates: 8, 11.025, 16, 22.05, 24, 32, 44.1, 48 kHz. It also seems fine at 12kHz. It supposedly supports variable rates from 8 to 50kHz as well, but this interface does not utilize that. You can check the actual rate with "cat /dev/sensors/vxp/status".
Configuring TimeSync
record_from also records sync info. This is most useful if you are recording timestamps that are correlated across nodes. To do this you need to make sure that "global time" is properly configured.
This is quite simple to do. Simply modify the .run file to make sure that this line is correct:
&gsync_link(uses=ls0, args="--root 104 --use_cpu_time");
You just need to make sure that the --root <ID> argument refers to a node that exists. On startup, each node will check to see if they are "the root", and if they are they will start broadcasting their own clock as "global time". If nobody broadcasts, global time won't work.
The simplest solution is to rename one of your nodes to be '104' (set the /etc/id file on that node to read '104'). Or, change the .run file to specify a node that you have.
A future version might find a better way to configure this, but.... well....
NOTE: Synchronization depends on having connected "triangles" of nodes, i.e. any three nodes that can all mutually hear each other will sync to each other, and adjacent triangles can also sync by transitivity. For well-connected networks this is rarely a problem, but in sparse networks, simpole connnectivity is not sufficient to guarantee sync connectity.
Synchronizing sampling across nodes
Syncronization in this system operates by conversion, i.e. a sample number on one node can be converted to the CPU clock on that node, and then converted to the CPU clock on another adjacent node, and then to the sample clock on that other node.
The easiest way to sync across nodes is to use the gsync (gps time) facility described above, in which a single node broadcasts its time as "global time" and via this conversion process all the other nodes can relate their clocks to it. If GPS time is present, then each node can convert a sample clock value to "gps time", which is a timestamp all the nodes can understand and convert to their own sample clock, etc.
The sync facility is easy to use from your software program. For example, if you want to do recordings all from the same time, you can write a program that listens for a control packet containing a time in "gps time", converts that time to its local sample clock and requests data starting at that sample. (Note, the sync system reports "scaled" sample clocks (20x) in order to enable conversions to and from fractions of samples; so you need to divide the converted time by 20).
You can also do this without writing any software. If you use record_from and add the arguments
--start_clock gps --start <a time in GPS time>
and on your "master node" you choose a start time ("now") using
gettime --clock gps
Then, you can then issue the command to run record_from using rbsh. Just make sure that you execute the "gettime" part on the master node only and pass the result via rbsh. Otherwise each node will execute gettime and they will all get slightly different times.
Note, that even if you don't start recordings at the same time, if you're using record_from you can use the .sync files to figure out the sync after the fact (if you're using record from with the proper args to save gps time stamps). The .sync file should contain everything you need to line up the signals afterwards. .sync is a text file containing the gps timestamps corresponding to sample indices. the first line refers to the start of the file, i.e. that sample index corresponds to the first sample in the .raw file, for example:
9260081 185.201620 952191237.876948 0 0 9500868 190.017360 952191242.893193 0 0 #col 1: sample index (first index corresponds with first sample in raw file) #col 2: sample index * 20 in timeval form (sec.usec) #col 3: gps time in timeval form #col 4: warn of discontinuities in the time series: number of glitches #col 5: total number of samples dropped during glitches (if any).
Note, glitches typically result from low memory, for example if you are unable to record to media fast enough and it's buffering data in ram.
Using the .sync information, you can line up the sampled data from different nodes and even scale it to match rates over long time periods. Sound card frequency sources are generally only accurate to about 50 PPM. (see http://www.leapsecond.com/pages/sound-1pps/ )
Sample Code
Using these sensor devices and timesync system from your program is not well documented but there is some sample code to help you.
sensors/util/record_from.c is one interesting example; it demonstrates accessing the sensor device "manually" (not using the sensor client library) and also demonstrates using timesync to record and print the timestamps. It's also an intersting example of using threads and cross-thread messages. The reason it is like this is I was trying everything I could to get it to "keep up", trying to make it as efficient as possible.. but then I realized that the problem had nothing to do with my code, but was simply bottlenecks in the output side, in the writes to the flash card and in the writes over NFS.
sensors/audiod/examples/streamaudio.c is a much simpler example that uses the sensor client library to get a stream of data back and print it. It does not do anything with timesync though, so is less of a good example from that perspective. More information on sensor client is found in the libdev/include/sensor_client.h file, and on time synchronization in the timesync/include/sync.h header file
There are also several projects that use sensor device, including acoustic ranging in devel/loc/ar and audio monitoring in devel/audioclient.
A little more on how time synchronized sampling works
Sensor device provides both buffered and streaming interfaces, with a continuous timebase. The buffered interface can be used to retrive a specific sample range (if those samples are still buffered). The streaming interface outputs the data in chunks, annotated with the sample index of the first sample in each chunk. The sample index is monotonic.
The different forms of audio server all strive to provide a buffered, continuous timebase. If data is lost, 0's will be inserted to make up the gap. When multiple channels are provided, the channels are indexed identically so that a sample of index X from two channels correspond to the same sampling time. The audio server also interfaces to the timesync system in order to create a method of conversion from CPU time to sample index and vice versa. Other components provide conversions from CPU time to other clocks, e.g. GPS or other nodes' CPU clocks.
You can explore sensor device from the shell:
$cat /dev/sensors/vxp/all
usage: Use echocat to write a command and read back a stream of responses
start=<sample-num>
count=<num-samples>
relative: interpre start sample relative to latest sample
trigger: request a single sample to be recorded now
Note: If start is not specified, defaults to 0; in relative mode this
means the the latest sample, otherwise means the easliest buffered sample.
If count is not specified, detaults to infinity (streaming mode).
Additional commands may be implemented by specific devices.
Examples:
relative:trigger:count=1 => triggers a new sample and reports it
relative:start=-3:count=6 => reports six samples, starting three samples ago
relative => start streaming from now
Info:
Data is Isochronous
Timehist-Buffer ID: 100
384000 valid samples: 20137111104 to 20137495103
This gives usage for the command syntax for accessing sensor data. You can see that this node has been running for days as it has very large sample indices (these are 64 bit numbers). You can also see that this driver has a buffer of 8 seconds at 48KHz.
The vxp driver also has a status output:
$cat /dev/sensors/vxp/status VXPocket sampling service, node 0.0.0.107 --------------------------------------- Sample rate: nominal=48000, observed=48001.90 Num samples read last: 1260 Sync to timehist on id 100 Current sample counter: 20141628156 Current codec offset: -703 Sync resets for noconv: 18 Sync resets for delta: 2 Total sync resets: 9 Full sync resets: 8 Pcm 0: fd=13, In Sync, 1220 in pre-sync buffer Sample offset: 7311891149 Next Jump: 0 Channel signal levels: 0/0 *** Channel 0 DEAD *** *** Channel 1 DEAD *** Pcm 1: fd=22, In Sync, 0 in pre-sync buffer Sample offset: 7311891852 Next Jump: 0 Channel signal levels: 0/0 *** Channel 2 DEAD *** *** Channel 3 DEAD ***
Much of this is technical or debugging information, but it does tell you the sample rate and the observed rate (according to CPU time). It also says that the channels are all 'DEAD'. This is because no batteries are plugged into the array.
You can use echocat, a very simple program that opens a device writes a string, and reads back the results, to get a text output of recent samples:
$echocat -w /dev/sensors/vxp/all relative:count=10
#Sample-Size: 8
#Sample-Num ch0 ch1 ch2 ch3
20148051383 -1 0 -1 -1
20148051384 0 0 -1 0
20148051385 -1 -1 -1 0
20148051386 0 0 -1 -1
20148051387 -1 0 0 -1
20148051388 -1 -1 0 0
20148051389 -1 0 -1 -1
20148051390 0 -1 -1 -1
20148051391 0 0 0 0
20148051392 -1 -1 0 0
This opens the device file /dev/sensors/vxp/all, and issues the command relative:count=10, which means, considering "now" as sample 0, give me the range from 0-9 samples. With no batteries we get 1 bit of random noise. (Note, you have to use -w and control-c out of echocat after the trace completes; you could write a special sensorcat program that did not have this property.)
Note, you can also cat single:
$echocat -w /dev/sensors/vxp/single relative:count=10
#Sample-Size: 2
#Sample-Num ch0
20158117271 0
20158117272 -1
20158117273 -1
20158117274 -1
20158117275 -1
20158117276 0
20158117277 0
20158117278 -1
20158117279 0
20158117280 -1
Timesync also has relations from that sample index number to CPU time. Note that since I only have one node running, I don't have any conversions other than for the sample clock:
$cat /dev/sync/params/status
Conversion Parameters Available at Node 107 (0x6b)
From node 107.cpu to:
107./dev/sensors/vxp/all (via 107) RMS 5.7usec, basex=951894409.024140,
offset=-951491307.371018, m=-0.039962, b=3475730.750000
From node 107./dev/sensors/vxp/all to:
107.cpu (via 107) RMS 6.2usec, basex=403105.128860,
offset=951491303.895280, m=0.041625, b=7.478168
This shows that it's possible to convert a 107.cpu time to 107's sensor clock (vxp/all.. shared with vxp/single), with an rms fit error of 6 usec.
You can use the gettime utility (a very simple program that also serves as an example) to get the current time in some other clock, for example the sample clock:
$gettime --clock /dev/sensors/vxp/all 403365.135219
All this does is call gettimeofday() and then convert it from CPU to the specified clock. So if you ask for --clock cpu, you get this:
$gettime --clock cpu 951894772.474005 $date Wed Mar 1 07:12:57 UTC 2000
Now you might wonder, why does gettime report 403365.135219 (seconds) shortly after i requested samples and they were reported as index 20158117280? The answer is that in order to enable wall time to be converted to fractions of a sample, vxp_server multiplies the sample clock by 20 (the scale parameter to record_from). so 20158117280 * 20 = 403162345600 = 403162.345600.. a few minutes before i ran gettime.
There are two solutions to time sync sampling. One is to broadcast a global time reference. If you have actual GPS units, this can be a good solution because you can broadcast them from several points and gsync will do the right thing, averaging the error at the boundaries. You can also use mutlihop timesync by integrating hop-by-hop conversion into your network layer. The flood daemon supports this, and this is what is used by acoustic ranging. Every time a message is received from a neighbor, any time stamps in the message are converted into the local timebase before the message is forwarded. In this way, when the message is received it can be converted from a neighbor's timebase into the local timebase and interpreted.
