| |
ZL3AI > APRDIG 03.02.03 10:05l 360 Lines 11848 Bytes #999 (0) @ WW
BID : 817-ZL3AI
Read: GUEST
Subj: TAPR Digest, Jan 29, 4/6
Path: DB0FHN<DB0ZWI<DB0HST<ON0DXC<WA7V<RZ6HXA<SP7MGD<VK7AX<ZL2BAU<ZL3VML
Sent: 030202/0907Z @:ZL3VML.#80.NZL.OC #:58659 [Chch-NZ] FBB7.00i $:817-ZL3AI
From: ZL3AI@ZL3VML.#80.NZL.OC
To : APRDIG@WW
Subject: New spec - working notes
From: Miller Scott Contr 30SCS/FTI <Scott.Miller@vandenberg.af.mil>
Date: Wed, 29 Jan 2003 22:57:41 -0000
X-Message-Number: 27
Thanks to everyone who responded off-list. I figure it's time we had
something concrete to fight about, so here's my proposal for a general
message header and two primitive message types.
Message Header Format
bits: 1 7 8 8
-------------------------------------------------------------
| ext_flag | length | msg_type | ext_type | message body... |
-------------------------------------------------------------
ext_flag - A '1' in this field indicates a 16-bit message ID.
length - Length of message, inclusive of message ID.
msg_type - The message ID if ext_flag = 0, otherwise the low byte of the
message ID
ext_type - High byte of message ID. Only present if ext_flag = 1.
This message type scheme allows for 65,792 message types (256 standard and
65,536 extended).
8-bit message IDs are reserved for common, standard messages. 16-bit IDs
are used for less-frequently used messages and experimental message types.
0-byte messages are supported for instances where the presence of the
message type alone conveys the required information.
No CRC is included, as a reliable transport layer is assumed.
All numbers are little-endian.
Position format:
All positions reference the WGS84 datum using decimal degrees.
Coordinates are represented by 32-bit unsigned integers as follows:
Integer longitude = (longitude + 180) * 2^23
Integer latitude = (latitude + 90) * 2^24
Altitude is in meters above WGS84 ellipsoid, stored as a 16-bit unsigned
integer.
Time format:
All times are in seconds elapsed since 1 Jan 1970 00:00:00.0 UTC, stored as
a 32-bit unsigned integer.
This is equivalent to the standard Unix time format.
MSG #01 - Position
[Latitude] 32 bits
[Longitude] 32 bits
[Alititude] 16 bits
Example:
0B 01 06 81 F5 7C 5E BA C9 1D 65 01 = 34.959 N, 120.424 W, 257 meters
MSG #02 - Timestamp
[Time] 32 bits
Example:
05 02 6E 3E 38 3E = Wed, 29 Jan 2003 20:49:50 UTC
Position with timestamp example (18 bytes total):
0B 01 06 81 F5 7C 5E BA C9 1D 65 01 05 02 6E 3E 38 3E
Errata
------
16-bit unsigned altitudes may be a problem in areas like Death Valley.
Use of signed integers would limit altitudes to 32,767 meters max.
This could pose a problem for balloon experiments at high altitudes.
----------------------------------------------------------------------
Subject: Re: New spec - working notes
From: James Jefferson <jjeffers@aprsworld.net>
Date: Wed, 29 Jan 2003 17:03:44 -0600
X-Message-Number: 28
>Errata
>------
>
>16-bit unsigned altitudes may be a problem in areas like Death Valley.
>Use of signed integers would limit altitudes to 32,767 meters max.
>This could pose a problem for balloon experiments at high altitudes.
64-bit numbers are going to be nescessary for altitude and time. The 32 bit
time stamp will overflow by 2038, so there really isn't much point in
putting an expiration label on a new protocol.
Altitude in the MIC-E can run from the deepest ocean on up, so we should
probably use an signed 32 bit number.
-Jim KB0THN
----------------------------------------------------------------------
Subject: EMAIL
From: Gonzales Andy <agonzales@arc.nasa.gov>
Date: Wed, 29 Jan 2003 15:10:36 -0800
X-Message-Number: 29
I am fascinated by the possibilities and how they might be useful for
my project. Right now, anything we do in the RF world is under the
supervision of a licensed HAM.
--
Andy Gonzales
NASA / Ames Research Center
Moffett Field, CA 94035-1000
Bldg. N-244, Rm. 240
MS 244-19
agonzales@arc.nasa.gov
650-604-0309
----------------------------------------------------------------------
Subject: New spec - short-term deliverables
From: Miller Scott Contr 30SCS/FTI <Scott.Miller@vandenberg.af.mil>
Date: Wed, 29 Jan 2003 23:14:27 -0000
X-Message-Number: 30
Here's what I'm hoping to produce in the near term:
1. An APRS-IS translator that'll translate a full stream and provide it to
clients in the new format.
This will probably be an NT service, unless someone wants to
volunteer to write a Unix daemon.
2. An ActiveX control to interface with the translator.
Just tell it to connect, and it'll raise an appropriate event
whenever a recognized message is received. If it doesn't recognize the
message format, it'll pass the raw data to a generic event so you can parse
your own messages.
I think I'll also add an encapsulation message type for carrying standard
APSR messages. That way, you can drop the control into an existing
application, and use your existing parser to handle those messages that
aren't translated yet.
I figure that if I can get this going, it really doesn't matter if anyone
else adopts it as a standard. It'll still be useful to me, and if the
translator becomes a standard feature on APRS-IS servers it'll make client
development easier and reduce bandwidth usage, at least a bit. Notice that
the example message took only 18 bytes for time and position with better
precision than current formats allow.
Scott / N1VG
----------------------------------------------------------------------
Subject: RE: New spec - working notes
From: Miller Scott Contr 30SCS/FTI <Scott.Miller@vandenberg.af.mil>
Date: Wed, 29 Jan 2003 23:36:18 -0000
X-Message-Number: 31
32-bit time only runs out in 2038 for 32-bit signed integers. Unsigned
should last us until 2106.
Yeah, I think 16 bits isn't going to be enough for altitude, but 64 bits is
kind of excessive. 32 bits would give us a range of roughly +/- 2 billion.
24 bits would be more reasonable at +/- 8 million. That'd still leave you
enough room to have, say, 4 bits of mantissa, for 1/16 meter resolution and
a range of +/- 500,000 meters.
Remember, we're trying to keep bandwidth requirements down, and make it easy
for microcontrollers to parse the basics.
Scott / N1VG
----------------------------------------------------------------------
Subject: RE: New spec - working notes
From: Miller Scott Contr 30SCS/FTI <Scott.Miller@vandenberg.af.mil>
Date: Thu, 30 Jan 2003 00:43:00 -0000
X-Message-Number: 32
Updated notes to include new altitude format, plus course and speed and
comment messages: (Bob, you're unusually quiet. Comments?)
N1VG format working notes
Message Header Format
bits: 1 7 8 8
-------------------------------------------------------------
| ext_flag | length | msg_type | ext_type | message body... |
-------------------------------------------------------------
ext_flag - A '1' in this field indicates a 16-bit message ID.
length - Length of message, inclusive of message ID.
msg_type - The message ID if ext_flag = 0, otherwise the low byte of the
message ID
ext_type - High byte of message ID. Only present if ext_flag = 1.
This message type scheme allows for 65,792 message types (256 standard and
65,536 extended).
8-bit message IDs are reserved for common, standard messages. 16-bit IDs
are used for less-frequently used messages and experimental message types.
0-byte messages are supported for instances where the presence
of the message type alone conveys the required information.
No CRC is included, as a reliable transport layer is assumed.
All numbers are little-endian. Signed integers use two's complement
notation.
---
MSG #01 - Position
[Latitude] 32 bits
[Longitude] 32 bits
[Alititude] 24 bits
Positions reference the WGS84 datum using decimal degrees. Coordinates are
represented by 32-bit unsigned integers as follows:
Integer longitude = (longitude + 180) * 2^23
Integer latitude = (latitude + 90) * 2^24
Altitude is in centimeters above WGS84 ellipsoid, stored as a 24-bit signed
integer.
Example:
0C 01 06 81 F5 7C 5E BA C9 1D 7C 47 00 (34.959 N, 120.424 W, 183.0 meters)
---
MSG #02 - Timestamp
[Time] 32 bits
Times are in seconds elapsed since 1 Jan 1970 00:00:00.0 UTC (UNIX format).
Example:
05 02 6E 3E 38 3E (Wed, 29 Jan 2003 20:49:50 UTC)
---
MSG #03 - Freeform Comment
[Text] Up to 126 characters, not zero-delimited
The purpose of the comment is to provide a free-form text field associated
with the originating station.
Example:
05 03 41 42 43 44 (ABCD)
---
MSG #04 - Course and Speed
[Course] 16 bits (degrees)
[Speed] 16 bits (cm/sec)
Speeds are in centimeters per second. Course is in degrees relative to true
north.
Example:
05 04 C3 00 AE 08 (195 degrees at 80 km/hr)
----------------------------------------------------------------------
Subject: APRS-IS Sysops - Potential Bottleneck Identified - Part 1
From: "AE5PL Lists" <HamLists@ametx.com>
Date: Wed, 29 Jan 2003 20:28:53 -0600
X-Message-Number: 33
This is sent in 2 parts for clarity.
Recently, it was brought to my attention that a large number of
duplicate packets had begun to appear on APRS-IS that were not getting
blocked by the servers. The investigation that followed produced some
information that I think may be of high importance to APRS-IS server
operators. I am only the conduit for the dissemination of this
information. I wish to express my gratitude to the person who
originally brought this to my attention by providing some very good and
specific feedback regarding strange track lines. I also wish to express
my gratitude to the sysop that spent extensive time and energy on
isolating and identifying the bottleneck described below.
For those that have worked with and monitored traffic on APRS-IS, I am
sure that many have noticed the data from one server or another becoming
bursty for no apparent reason. Sometimes this burstiness would cause
"quiet periods" of 20 seconds and more followed by huge (64k or more)
chunks of packets suddenly appearing. With an average data rate of 1000
bytes per second (actually, the full APRS-IS feed is higher than this
but 1000 will work well for this example), a 64k chunk of data
represents over 60 seconds of packets being delivered all at once.
Hence, duplicate packets which are legitimately received over 30 seconds
apart might show up to the next server within one second of each other
at the next server down the line. The other situation is where
duplicate packets span the gap because they are received from multiple
sources, one demonstrating the bursty transmission, the other with a
"smooth" transmission. This situation could cause duplicate packets
that normally would be suppressed to now appear on APRS-IS. This is a
wordy way of saying that the burstiness seen on APRS-IS was actually
causing significant delays in the delivery of packets, thereby defeating
dupe check algorithms.
There has been a large amount of exploration into the source(s) of these
delays. We looked at applications, operating systems, computer
hardware, network hardware, and configurations. Through the diligent
efforts of others, each of these was examined and potential bottlenecks
were identified and/or eliminated. The single bottleneck found that
could account for this extended burstiness is something known as
"throttling", "load shaping", or "traffic shaping" on the link between
the server and the Internet (this probably has other names, but I will
use the term "throttling" for brevity). What was especially surprising
about this was the fact that testing revealed that throttling on the
inbound side (to the server) of the link (normally less than 1/2 of the
bandwidth of the outbound side) was actually affecting the outbound side
to a much greater degree than the inbound side. Without going into a
lot of detail on how TCP/IP works, suffice it to say that acks were
getting dropped causing excessive retransmissions.
Continued in part 2...
Pete Loveall AE5PL
pete@ae5pl.net
----------------------------------------------------------------------
Read previous mail | Read next mail
| |