README 7.92 KB
Newer Older
1
Welcome to Multicat!
2
====================
3 4 5 6 7 8

The multicat package contains a set of tools designed to easily and
efficiently manipulate multicast streams in general, and MPEG-2
Transport Streams (ISO/IEC 13818-1) in particular.

The multicat tool itself is a 1 input/1 output application. Inputs and
9
outputs can be network streams (unicast and multicast, IPv4 and IPv6), files,
10 11
directories, character devices or FIFOs. Typical applications are recording
live transport streams, or playing out TS files without modification.
12 13 14

Multicat tries to rebuild the internal clock of the input stream; but
it wants to remain agnostic of what is transported, so in case of files
15 16 17 18
or directories, the said clock is stored to an auxiliary file (example.aux
accompanies example.ts) while recording. Other inputs are considered "live",
and the input clock is simply derived from the reception time of the
packets.
19 20 21 22 23 24 25

IngesTS is a companion application designed to manipulate TS files. It
reads the PCR values of the file, and builds the auxiliary file that is
necessary for multicat.

OffseTS is another companion application to manipulate auxiliary files.
Given an offset in time from the beginning of the file, it returns the offset
26 27
of the position in number of packets. It is currently deprecated in favour of
using the -k and -d options of multicat.
28

29 30 31
LasTS is also a companion application which gives the total duration of an
auxiliary file.

32
Finally aggregaRTP and reordeRTP can be used to carry a high-bitrate
33 34 35 36
signal over one or several contribution links, and support retransmission
of lost packets via an additional UDP or TCP connection. ReordeRTP can also
smooth up the reception of a stream from a link that is known to reorder
and add jitter to packets.
37 38 39

The multicat suite of applications is very lightweight and designed to
operate in tight environments. Memory and CPU usages are kept to a minimum,
40
and they feature only one thread of execution.
41

42 43 44 45 46 47 48

The socket description format
=============================

For conveniency all tools use the same way of describing a socket in a
program argument:

49
<connect address>:<connect port>@<bind address>:<bind port>/<options>
50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66

All parts are optional; default port (1234) or wildcard address will then
be used.

Some examples:

Reading all streams coming to port 5004:
    @:5004
Reading from a multicast stream on port 5004:
    @239.255.0.1:5004
The same, with source-specific multicast:
    192.168.0.1@239.255.0.1:5004
Writing to a multicast stream on port 5004:
    239.255.0.1:5004
The same, but binding to a specific interface:
    239.255.0.1:5004@192.168.0.2

67 68 69 70 71
Options include:
 /ifindex=X (binds to a specific network interface, by link number)
 /ifaddr=XXX.XXX.XXX.XXX (binds to a specific network interface, by address)
 /ttl=XX (time-to-live of the UDP packet)
 /tos=XX (sets the IPv4 Type Of Service option)
72
 /tcp (binds a TCP socket instead of UDP)
73 74
 /srcaddr=XXX.XXX.XXX.XXX (source address for raw packets)
 /srcportr=XX (source port for raw packets)
75

76 77 78 79 80 81 82 83
Example:
    239.255.0.1:5004/ttl=64

Also, multicat supports IPv6 addresses.


Basic uses
==========
84 85 86 87 88 89 90 91 92 93 94 95 96 97 98

Recording a multicast address to a file:

multicat @239.255.0.1:5004 /tmp/myfile.ts

This will also create file /tmp/myfile.aux. Playing back the file:

multicat -p 68 /tmp/myfile.ts 239.255.0.2:5004

Adding an RTP header to an existing multicast stream:

multicat -p 68 -u @239.255.0.1:5004 239.255.0.2:5004

The PCR PID is here supposed to be 68. If you don't specify it, the timestamps
will not be RFC-compliant (but will work in most cases). You can use the
99 100 101
mpeg_print_pcr example from biTStream to determine it. Otherwise, if you are
sure the stream contains a single program, and only one PID carries a PCR, you
can pass "-p 8192" the disable the PID check. This isn't on by default because
102 103 104
it can produce awful things with multi-program transport streams, and the
world would be a better place if people had to knowingly turn it on.

105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120
Starting at a given position for a given duration:

multicat -p 68 -k 270000000 -d 2700000000 /tmp/myfile.ts 239.255.0.2:5004

In this case multicat discards the first 10 seconds, and only plays the file
for 100 seconds. Internally multicat uses a 27 MHz monotonic clock, and all
offsets (starting from 0) and durations are expressed in this unit.

Making an extract of a recorded file to a plain TS file:

multicat -f -k 270000000 -d 2700000000 /tmp/myfile.ts /tmp/extract.ts

The option -f allows outputting the extract as fast as the output item can
write it, without reproducing the same pace as the original stream (and thus,
waiting 100 seconds).

121 122 123 124 125 126 127 128 129 130 131 132 133 134

Using IngesTS
=============

ingests -p 68 /tmp/afile.ts

This will create file /tmp/afile.aux. 68 is supposed to be the PCR PID.
The same note as above applies to ingesTS.

Playing the file:

multicat -p 68 /tmp/afile.ts 239.255.0.2:5004


135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162
Working with directories
========================

Starting with version 2.0, multicat can write or read a continuous stream to
discontinuous buffers, while still retaining the original properties. For
instance:

mkdir mydir
multicat @239.255.255.1:5004 mydir

creates file XXXXXX.ts and XXXXXX.aux in mydir. Every hour, multicat closes
both files, and creates (XXXXXX+1).ts and (XXXXXX+1).aux. And so one. The
stream can then be identically replayed with a 100 seconds delay with:

multicat -p 68 -k -2700000000 mydir 239.255.255.2:5004

A negative value to -k implies "from the end", in this case from the present
time. To make an extract of the stream:

multicat -f -k 35383033980000000 -d 27000000000 mydir extract.ts

With the directory input/output, timestamps represent the number of ticks of
a 27 MHz real-time clock since the 1st of January 1970 (UNIX Epoch). It is
therefore possible to pass absolute (positive) dates to -k.

There is no built-in expiration of files in multicat; to avoid filling up the
partition, it is necessary to run multicat_expire.sh every hour.

163 164 165 166 167
The duration of the segments may be specified with -r. It is also advised to
add an offset with -O (typically a per-stream random number of up to the
segment duration minus one) to avoid having all multicat processes rotate
files exactly at the same time, resulting in a surge in CPU usage and disk I/O.

168

169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189
Using OffseTS
=============

We want to take the first 60 minutes of a TS file. We must scale it in a
27 MHz clock:
60 * 60 (seconds) * 27000000 (MHz) = 97200000000

Find the offset in 1316-blocks:

offsets /tmp/myfile.aux 97200000000

It returns for instance "556896". Then cut the file using dd:

dd if=/tmp/myfile.ts of=/tmp/mynewfile.ts bs=1316 count=556896

Alternatively, if we want to *remove* the first hour:

dd if=/tmp/myfile.ts of=/tmp/mynewfile.ts bs=1316 skip=556896

It can also be done with multicat using the -s and -n options.

190 191 192 193
OffseTS is currently deprecated in favour of using the -k and -d options of
the multicat program. OffseTS is still distributed for compatibility, but
doesn't support the new directory input.

194

195 196
Using AggregaRTP and ReordeRTP
==============================
197 198 199 200 201 202 203

Splitting an RTP stream to two streams with different routing policies:

aggregartp @239.255.0.1:5004 239.1.0.1:5004@192.168.0.1 239.2.0.1:5004@172.16.0.1

At the other end, reassembling the two streams into one usable stream:

204 205 206 207 208 209 210 211 212 213 214 215 216
reordertp 192.168.0.1@239.1.0.1:5004 172.16.0.1@239.2.0.1:5004 239.254.0.1:5004

Transmit a signal over a lossy link:

aggregartp @239.255.0.1:5004 239.1.0.1:5004 -X @:5006
reordertp @239.1.0.1:5004 239.254.0.1:5004 -X 192.168.0.1:5006

The same, but using inverted TCP for retransmission (for NAT traversal for
instance):
reordertp @239.1.0.1:5004 239.254.0.1:5004 -X @:5006/tcp
aggregartp @239.255.0.1:5004 239.1.0.1:5004 -X 192.168.0.2:5006/tcp

(with TCP the listener must be started before the other)
217 218 219 220 221 222 223 224 225 226 227 228 229


Using multilive
===============

Running a master at priority 1000 on multicast address 239.255.255.255:1025:

multilive -y 1000 @239.255.255.255:1025 239.255.255.255:1025

Running another master on a different machine at a higher priority (who will
preempt the other master):

multilive -y 1001 @239.255.255.255:1025 239.255.255.255:1025