Re: OPRA is not FAST
Daniel May / SpryWare, LLC <>
7 Nov 2008 10:42PM ET
Shwetang,
While others have used a proprietary header and trailer, there are ways to accomplish what you are trying to do with a fully FAST compliant message. Greg Maynard (gmaynard@ise.com) of the ISE brought this forward a few months ago, and we came up with a "best practice" for defining a frame header with exactly the information you are trying to convey.
The idea is to use a FAST Byte Vector to carry a network byte order sequence number at the beginning of each frame. This will give a sniffer a predictable, easy to find header and sequence number in an easy to read format. The rough draft of the message looked like this:
C0 F8 = pmap + Fast reset message
F0 = pmap (TID+3 bits)
F7 = TID 119,
84 = Byte Vector size=4 (with high bit set)
00 00 00 01 = data=1 (frame seqno, uint32_t range 0 -4,294,967,295)
82 = byte vector size=2 (with high bit set)
03 E6 = data=998 (frame length, uint16_t range 0 - 65,535)
84 = byte Vector size=4 (with high bit set)
02 E2 D3 88 = byte vector data=48419720 (frame sending time, uint32_t, milliseconds since epoch)
So what you have is a standard frame header that contains a Reset Message, and using template ID 119, 3 byte vectors containing a frame sequence number, frame length, and sending time. Obviously, some of these fields are optional, but useful. You could always add things like messages per frame, etc. The fact that the header is fixed in size means you can efficiently populate it *after* frame construction, and move the values into the output buffer just prior to writing to the network. You can also discard a frame during arbitration very quickly, or skip to the next frame in a continuous stream of data.
I will contact Greg to see if we can get the exact "Block Header" that the ISE has chosen, along with the template definition so we can start effectively using it as a "best practice".
I will let Rolf follow up on item #2
/Daniel
>
>
> For the point # 1. As far as I understood the specification of FAST1.1,
> proprietary header and trailer is completely independent of the
> compliance. This is application specific and does not play any role for
> the FAST (like UDP packet header has nothing to do with FAST). More over
> the greatest advantage of the packet header is for the network group who
> wants to check the sequence of the messages without decoding the FAST
> messages. This header give flexibility and information about the
> sequence number of the 1st message and the number of messages/packet,
> size of the message etc. This information will be vary valuable for the
> tools like Sniffer where without integrating FAST protocol to tools
> network engineer can analyze the gaps in the packets. Actually FAST
> working group should consider this idea for the future.
>
> For the point # 2. OPRA consist of generalize template for all the
> messages. FAST engine can (must) run without any business knowledge of
> the message and strictly based on the information received from the
> wire. So if the FAST 1.1 compliant decoder is placed with these
> messages, it should work. Later application needs to build the state
> engine to constitute the message based on the information received from
> the FAST engine(decoder/template).
>
> Sample decoder is given as an example and to reconstitute the
> original message to ASCII. State machine is being used to decode the
> original message.
>
> Now going back to the original comments about the compliance. OPRA uses
> only two different type of operators.
> 1. Copy : There is no change from FAST 1.0 to FAT 1.1 spec. So this
> operator for 1.0 is comply with 1.1
> 2. increment : There is no change from FAST 1.0 to FAT 1.1 spec. So this
> operator for 1.0 is comply with 1.1
>
> So if the reference for adding more template IDs for bbo indicator then
> it is incorrect as per the spec as it will make FAST engine from
> generalize engine to OPRA specific.
>
>
> I am still looking in the section 10.0 of FAST 1.1 where there are some
> ambiguity which needs to be resolve. For example, the xml statement
> doesn’t mandate to have the templateIdentifier as the 1st bit of the
> presence map. message ::= segment segment ::= PresenceMap
> TemplateIdentifier? (field | segment)*
>
> and the other statement where TID is optional as the 1st bit.:
>
> “A segment has a header consisting of a Presence Map followed by an
> optional Template Identifier. The segment has a template identifier
> either if it is a message segment, or if the segment appears as the
> result of a dynamic template reference instruction. A template
> identifier is encoded as if a copy operator was specified. The operator
> uses the global dictionary and has an internal key common to all
> template identifier fields. This means that a segment with a template
> identifier does not always contain the template identifier physically.
> However, the first bit in the presence map is allocated by its copy
> operator. “
>
> Please let me know if I am misinterpreting the spec.
>
> -Shwetang
@FIXProtocol
