Button "Show recent activity"
Mahesh Kumaraguru <>
12 Jul 2011 6:39AM ET
Button "Show recent activity"
Hi,
Long back, I had asked "Forums message display enhancement request to highlight recent activity in forum" at
http://fixprotocol.org/discuss/read/ac424324
This find any of "a e i o u" can be used to show recent activity within any specific forum. In any discussion forum when search for any of the terms "a e i o u" is done, the messages in the search results is shown sorted in time order by post. For example, in the High Performance Interface WG's forum at
http://fixprotocol.org/discuss/search/149
when I searched for any of "a e i o u", the results I got is
- - - - - - - - - - - - - - - - - - - - - - - - -
[Start of Search results]
- - - - - - - - - - - - - - - - - - - - - - - - -
Executions for different orders belonging to the same list
Mahesh Kumaraguru 22 May 2011 9:51PM ET
Hi All,
Re: Requirements
John Harris / BondMart Technologies, Inc. 6 Mar 2011 10:13AM ET
Thank you, Rolf. Let me just summarize and respond to those aspects of your reply that appear to require further response from me. I hope others will share their thoughts as well.
Re: Requirements
Rolf Andersson / Pantor Engineering 6 Mar 2011 5:09AM ET
>>I believe we are expected to relate to the existing dictionary.
Re: Requirements
John Harris / BondMart Technologies, Inc. 5 Mar 2011 11:12AM ET
Your edits were fine - thank you - and your questions and comments are helpful. Please see my responses inline as well.
Re: Organizing the Work of Writing Requirements
Rolf Andersson / Pantor Engineering 5 Mar 2011 4:58AM ET
the FIXwiki currently has no direct relationship to the HPIwg (or any other wg afaik). This may change in the future but it has not been discussed to my knowledge.
Re: Organizing the Work of Writing Requirements
Rolf Andersson / Pantor Engineering 5 Mar 2011 4:49AM ET
John, having spent some time reviewing the dot-p2p.org wiki I remain unconvinced. There is little focus, a fair amount of random comments and some outright spam.
Re: Requirements
Rolf Andersson / Pantor Engineering 5 Mar 2011 2:38AM ET
I have edited your post in an attempt to create a response that is short and concise. Apologies if I have removed or changed anything that has bearing on my response. Please find my comments inline.
Requirements
John Harris / BondMart Technologies, Inc. 4 Mar 2011 2:36PM ET
The data dictionary contains over nine thousand lines of descriptive information, including over five hundred sets of enumerated values. Some of these sets have over one hundred members.
Re: Organizing the Work of Writing Requirements
John Harris / BondMart Technologies, Inc. 2 Mar 2011 3:32PM ET
As a contributor to the effort, I would like to try an open wiki as a development tool for the new protocol (while freely admitting that it may prove impractical or unproductive). My understanding is that participants generally supported the idea of open participation - and an open wiki - at the first meeting of what is now the High Performance Interface Working Group. In order to be as successful as possible, an undertaking such as this has to seek ideas and help from all over the world; it has to be open to experimentation and failure; it has to challenge pre-conceived notions. At the same time, it has to have some organization to it, though that organization needn't be centrally coordinated or hierarchical.
Re: Organizing the Work of Writing Requirements
Mahesh Kumaraguru 2 Mar 2011 1:22PM ET
I sent an account request for the newly released FIXwiki at
Re: Organizing the Work of Writing Requirements
John Harris / BondMart Technologies, Inc. 2 Mar 2011 12:19PM ET
Further to this post, I promised to post an example of a protocol that is being organized through a wiki. The one I had in mind is dot-p2p, which is a new, peer-to-peer DNS:
Organizing the Work of Writing Requirements
John Harris / BondMart Technologies, Inc. 2 Mar 2011 11:57AM ET
As discussed on our call this morning, I would like to see this group try an open wiki to help with the process of documenting requirements, which is the next task at hand.
Internet Open Trading Protocol @ IETF.Org
Mahesh Kumaraguru 26 Feb 2011 1:54AM ET
My Google search for "Open Trading Protocol" led me to Internet Open Trading Protocol @ IETF.Org
Re: Scope of HFT working group in terms of the phases of trade life cycle expected to be covered
John Harris / BondMart Technologies, Inc. 18 Feb 2011 5:59AM ET
Thanks, Mahesh. My replies are preceded with +++.
Re: Scope of HFT working group in terms of the phases of trade life cycle expected to be covered
Hanno Klein / Deutsche Börse Systems 18 Feb 2011 2:56AM ET
> [...] My view is that the future architecture would be diverging with Market Data over FAST over UDP, Trade Messaging over HFT over TCP and Settlement / post trade activities happening using FIXML over MQ.
Scope of HFT working group in terms of the phases of trade life cycle expected to be covered
Mahesh Kumaraguru 18 Feb 2011 2:25AM ET
Thanks for your feedback. I am renaming this branch thread since we have started discussing the scope of our working group in terms on the phases of trade life cycle expected to be worked on by our presently named "High Frequency Trading" working group.
Re: Compiling use cases / test cases for HFT
John Harris / BondMart Technologies, Inc. 17 Feb 2011 2:17PM ET
Trading is one phase or activity in portfolio or market-position management. Imagine that a trader has reached a satisfied state. He takes in new information. This information unsettles him. He evaluates whether he should act. He decides to act. He submits an order. He receives order-state information. If executed, he settles his obligation and returns, however briefly, to a satisfied state. Repeat process.
Re: Compiling use cases / test cases for HFT
Mahesh Kumaraguru 17 Feb 2011 11:30AM ET
You said "we will need trading firms and exchanges to verify the validity of the test data sets". Would FPL member exchanges, brokers, buy side firms and others be able to provide us with their valid "public testcases" and any other relevant data. Then our WG can find set of usecases / testcases common to all and various stakeholders can validate this common set.
Re: Compiling use cases / test cases for HFT
John Harris / BondMart Technologies, Inc. 17 Feb 2011 9:25AM ET
Starting with a blank sheet is best. My initial list of elemental use cases:
Re: Where is a good place to get the status of this effort?
John Harris / BondMart Technologies, Inc. 17 Feb 2011 6:09AM ET
If the goal is a widely-adopted, as-efficient-as-possible protocol for trade-related messaging, the sooner an open wiki or similar tool is launched, the better. Then, scope, requirements, proposals, etc. can be hashed out openly and quickly, around the clock. Let everyone play. Over 100 people have volunteered for this effort. I'm sure hundreds more would contribute if given the chance.
Re: Compiling use cases / test cases for HFT
Rolf Andersson / Pantor Engineering 16 Feb 2011 11:59PM ET
real life transaction traces are very valuable but hard to come by for obvious reasons. The next best thing is to synthesize traces using a statistical distribution gathered from real life data. This approach has lower fidelity and requires more work to verify the validity of the generated data. A third approach is to use public data as a basis and then "enrich" the data with synthesized data such as client ids, order ids etc.
Re: Where is a good place to get the status of this effort?
Rolf Andersson / Pantor Engineering 16 Feb 2011 11:52PM ET
Hi Andy, good to hear from you!
Re: meeting of the 25th of January
Rolf Andersson / Pantor Engineering 16 Feb 2011 11:45PM ET
no face to face meeting scheduled but still being considered. There will be two tech sessions related to the HFTwg at the FPL conference on March 1st.
Where is a good place to get the status of this effort?
Andy Faibishenko / Lasalle Technology 16 Feb 2011 11:34PM ET
I am trying to find out how far along the HFT FIX spec is - is there any kind of timeline for the first release? Also are there any working documents other than this discussion group?
Re: Request for Comment: Central Instrument Registry
John Harris / BondMart Technologies, Inc. 15 Feb 2011 9:24AM ET
With the exception of commodities, every financial instrument is uniquely defined by a legal instrument or set of legal instruments.
Re: Request for Comment: Central Instrument Registry
Jan Jonsson / E. Ohman J:or Fondkommission AB 14 Feb 2011 2:29PM ET
The problem with missing a common symbol to identify instrument is real, and important.
Re: Ability to transport multiple HFT application objects in a single HFT Protocol transmission message.
Mahesh Kumaraguru 14 Feb 2011 2:06PM ET
Thanks John. Its a please discussing these topics with you. Hope to have more fun as we work on FIX Protocol's different initiatives.
Re: Ability to transport multiple HFT application objects in a single HFT Protocol transmission message.
John Harris / BondMart Technologies, Inc. 14 Feb 2011 7:37AM ET
VALOMESS will be hard to top. Fortunately, my wife does not read this forum. Otherwise, she will know that other wives have a much better deal :-)
meeting of the 25th of January
Garry O'Reilly / Bolsas y Mercados Españoles (MEFF) 14 Feb 2011 5:57AM ET
I missed the last meeting. What was decided? When is the next meeting? Are we having a face to face meeting at the start of March?
Re: Ability to transport multiple HFT application objects in a single HFT Protocol transmission message.
Mahesh Kumaraguru 14 Feb 2011 5:15AM ET
Please see my comments inline,
Re: Ability to transport multiple HFT application objects in a single HFT Protocol transmission message.
John Harris / BondMart Technologies, Inc. 11 Feb 2011 8:57AM ET
Please see my comments (preceded with *** for clarity) in-line. I suspect that I may be misunderstanding your proposal, because you are arguing so passionately for it and I do not see the benefit at all. So, I welcome better understanding and to be proven wrong. Sorry to be slow in commenting.
Compiling use cases / test cases for HFT
Mahesh Kumaraguru 11 Feb 2011 1:44AM ET
What could we use as the starting point for compiling usecases / testcases for HFT? Could we start with testcases in one of the appendices of FIXProtocol specifications. Or should we start with a blank sheet because HFT design is being done from scratch rather that incrementaly refining / enhancing existing Tag=Value^ FIX and many new / different usecases could be found by taking a "from scratch" approach.
Re: Ability to transport multiple HFT application objects in a single HFT Protocol transmission message.
Rolf Andersson / Pantor Engineering 11 Feb 2011 12:24AM ET
as I commented earlier; it is unclear to me what benefits we can get from using a message structure like the one we have discussed here.
Definitions of related in business versus technical context
Mahesh Kumaraguru 10 Feb 2011 10:54PM ET
As per trading and/or market practices / FIXProtocol specifications what all could be considered as related in business context ? It would be really helpful if someone could elaborate more on this or point me to relevant documents at any website which could make this distinction clearer to me.
Re: Ability to transport multiple HFT application objects in a single HFT Protocol transmission message.
Mahesh Kumaraguru 10 Feb 2011 10:52PM ET
Below are my thoughts for each of the points you mentioned.
Re: Ability to transport multiple HFT application objects in a single HFT Protocol transmission message.
Hanno Klein / Deutsche Börse Systems 9 Feb 2011 5:10AM ET
I disagree. Just because messages come from the same counterparty does not constitute a business context. It is a technical context. FIX application level message types are primarily business functions. A new message type for batching blurs the line between application and transport. I am assuming you would want a repeating group of messages starting with the FIX field MsgType and not something along the line of the session layer message XMLnonFIX.
Re: Ability to transport multiple HFT application objects in a single HFT Protocol transmission message.
Mahesh Kumaraguru 9 Feb 2011 1:46AM ET
The application objects being batched together are related to each other in the business sense in that they originate from the same counterparty X and are destined to the same counterparty Y and hence they share the same routing information at HFT engine messaging level, so batching them into a single protocol transmission unit is correct in the "business sense".
Re: Request for Comment: Central Instrument Registry
John Harris / BondMart Technologies, Inc. 8 Feb 2011 10:39AM ET
Rolf and Mark have provided good leadership in beginning this effort with a focus on scope. The shared perception of a problem seemingly affecting many people and firms (perhaps fairly stated as, "FIX as it exists today may be less than optimal for high-frequency trading; doing something about that may be worthwhile") provides the motive force for the formation and efforts of the working group. Upon further reflection, it becomes obvious that market participants other than so-called "high-frequency traders," e.g., electronic market makers, may also benefit from the effort; that much has changed since FIX as we know it was created, and that if we were to set out on the same mission that Fidelity and Solly began almost twenty years ago, we might attack the problem(s) differently. At the same time, members of the working group appear to share the view that their efforts are not about "fixing FIX" - a protocol that has served the market extremely well and will continue to do so for years to come - but seeing what good they can accomplish starting from a clean sheet, looking with fresh eyes at the problems that are amenable to an open, standard, trading protocol.
Re: Ability to transport multiple HFT application objects in a single HFT Protocol transmission message.
Hanno Klein / Deutsche Börse Systems 8 Feb 2011 9:46AM ET
I believe we are on the same page. Your example was about "batched" order entry. With such a use case in mind for quotes, FIX has provided the MassQuote message and would provide a MassOrder if needed. There are always some common business elements, in your case e.g. "the same product" which get mapped to root level fields. Batching is something else for me as it is a technical term. The "batching device" has no business context, it just takes multiple independent messages and puts them physically together, possibly applying efficiencies related to repeating elements. The FIXML attribute does that (without efficiencies) and the FAST protocol or zlib do that (with efficiencies).
Re: Request for Comment: Central Instrument Registry
Hanno Klein / Deutsche Börse Systems 8 Feb 2011 9:25AM ET
I am in agreement with you that synthetic security IDs are good for performance. My statement was that the call for global IDs is made by people who have bigger problems than performance, i.e. the problem of uniquely identifying a tradable entity in the first place. Venues have moved to solve the performance problem by introducing their own synthetic IDs which are obviously only useful when trading on this one venue. A single synthetic ID is useful when trading on multiple venues and I agree again that an organization like FPL is well positioned in general to be an issuer of such a global ID. FPL has done so with IDs for marketplaces which have now become obsolete with ISO MIC values.
Re: Ability to transport multiple HFT application objects in a single HFT Protocol transmission message.
Donald Mendelson / CME Group 8 Feb 2011 9:19AM ET
> I agree and would like to add that batching only makes sense in the context of single business events which are typically reflected in FIX with a single message layout containing repeating groups, e.g. ...
Re: Request for Comment: Central Instrument Registry
John Harris / BondMart Technologies, Inc. 8 Feb 2011 8:44AM ET
I was pleased to learn of Bloomberg's Open Symbology Initiative, but find it deficient for the purpose at hand in multiple respects. Out of respect for what is a commendable effort, I won't enumerate here what I see as the defects but am happy to share them with you off-line.
Re: Ability to transport multiple HFT application objects in a single HFT Protocol transmission message.
Hanno Klein / Deutsche Börse Systems 8 Feb 2011 8:15AM ET
I agree and would like to add that batching only makes sense in the context of single business events which are typically reflected in FIX with a single message layout containing repeating groups, e.g. . If a new mass quote entering the book leads to updates of multiple price levels in multiple series, this single event can be efficiently broadcast in a single physical message (containing multiple logical messages) with FAST. This does not apply to a collection of independent business events, e.g. a mass entry of orders for different instruments that are sent to different matching engines and may or may not get filled. The engines run in parallel for performance reasons and send their results back to the submitter via a gateway holding the connection to the submitter. Waiting on the gateway for results from multiple sources to allow batching most likely defeats the purpose of high performance if you introduce delays for the sake of batching.
Re: Request for Comment: Central Instrument Registry
Hanno Klein / Deutsche Börse Systems 8 Feb 2011 8:00AM ET
We need to be aware of the fact that there are other "open" initiatives that try to solve the same problem, e.g. http://bsym.bloomberg.com/sym/. What is the incentive to support the service offered by FPL compared to other offerings?
Request for Comment: Central Instrument Registry
John Harris / BondMart Technologies, Inc. 7 Feb 2011 3:58PM ET
A useful complement of the new protocol would be a Central Instrument Registry to be maintained by FPL or another entity.
Re: Ability to transport multiple HFT application objects in a single HFT Protocol transmission message.
Rolf Andersson / Pantor Engineering 7 Feb 2011 2:37PM ET
John, you make a good point.
Re: Ability to transport multiple HFT application objects in a single HFT Protocol transmission message.
John Harris / BondMart Technologies, Inc. 7 Feb 2011 1:17PM ET
I can imagine that the ability to batch like or related objects, e.g., orders, in a single message may be valuable, but I am having trouble understanding the business case for batching seemingly unrelated objects.
Re: Ability to transport multiple HFT application objects in a single HFT Protocol transmission message.
Mahesh Kumaraguru 6 Feb 2011 12:15PM ET
My ideas are as below :-
Re: Language bindings for apps wtitten in C, C++, Java...
Rolf Andersson / Pantor Engineering 6 Feb 2011 2:24AM ET
the protocol should certainly be independent of a specific environment, but given that we are aiming for high performance I believe there is good reason to keep an eye on the effects of the various protocol design decisions.
Re: Ability to transport multiple HFT application objects in a single HFT Protocol transmission message.
Rolf Andersson / Pantor Engineering 6 Feb 2011 2:01AM ET
To make your proposal more complete, what are your definitions of:
Ability to transport multiple HFT application objects in a single HFT Protocol transmission message.
Mahesh Kumaraguru 5 Feb 2011 9:54PM ET
HFT Application Objects represent IOIs, Orders, Order Cancels, Order Cancel-Replaces, Executions, DKTs, Allocs etc. i.e. they are Application / non-Session objects exchanged between a buy side and a sell side. HFT Message - the basic unit of information exchange between counterparties is a sequence of bytes (the wire representation) used to facilitate exchange of application objects between these applications.
Re: Language bindings for apps wtitten in C, C++, Java...
John Harris / BondMart Technologies, Inc. 3 Feb 2011 11:55AM ET
I interpreted that section of the draft to be more of a question than a declaration - asking, in effect, what considerations/optimizations are applicable to the protocol-development effort.
Language bindings for apps wtitten in C, C++, Java...
Mahesh Kumaraguru 3 Feb 2011 11:11AM ET
Page 9 / 11 of Version 0.2 Scope definition Strawman document dated 15-Dec-2010 states :-
Re: Easily processed message - no overhead in accessin message content
Mahesh Kumaraguru 1 Feb 2011 10:31PM ET
You weren't terse, sorry I was thinking out aloud (just being tooo clear to myself :-)
Re: Request for Comment: Lightweight Order Message
John Harris / BondMart Technologies, Inc. 28 Jan 2011 11:33AM ET
Thank you, Peter. The technical objections are appreciated, too :-) Comments below:
Re: Request for Comment: Lightweight Order Message
Peter Franzen / Kanonkod 28 Jan 2011 9:41AM ET
> I will posit that it is possible for us to create an order message with just three business-application-level fields:
Re: Request for Comment: Lightweight Order Message
John Harris / BondMart Technologies, Inc. 28 Jan 2011 9:07AM ET
> I mean in the traditional sense of protocol layering (google for "protocol layering benefits" or similar). There are situations where mixing layers to gain performance makes sense, but in this case you are suggesting that about 32 bits worth of information is saved and it does not seem that you have exhausted other means for saving that data on the wire (just use FAST).
Re: Request for Comment: Lightweight Order Message
John Harris / BondMart Technologies, Inc. 28 Jan 2011 9:05AM ET
We could add to that list "buy to open," "buy to close," "sell to open," and "sell to close." But let us acknowledge that not all markets need all such constructs. We have no "sell short" orders in the bond market. We do sell short, but we do not mark our orders that way.
Re: Request for Comment: Lightweight Order Message
John Harris / BondMart Technologies, Inc. 28 Jan 2011 9:02AM ET
Thank you, Philip. Please see my comments in-line:
Re: Request for Comment: Lightweight Order Message
Anders Furuhed / Pantor Engineering 28 Jan 2011 8:44AM ET
I mean in the traditional sense of protocol layering (google for "protocol layering benefits" or similar). There are situations where mixing layers to gain performance makes sense, but in this case you are suggesting that about 32 bits worth of information is saved and it does not seem that you have exhausted other means for saving that data on the wire (just use FAST).
Re: Request for Comment: Lightweight Order Message
John Harris / BondMart Technologies, Inc. 28 Jan 2011 8:21AM ET
Anders,
Re: Request for Comment: Lightweight Order Message
John Harris / BondMart Technologies, Inc. 28 Jan 2011 8:13AM ET
Assume I will be sending this lightweight order message to an address that is guaranteed by the exchange to be unique as to instrument and side. Accordingly, by agreement, conveying instrument ID and side down the wire is unnecessary. The receiver presumes all messages arriving at that address are, e.g., buy orders for common shares of XYZ. Further, assume I am sending this message from a general-purpose computer with an operating system that knows how to send messages to that unique exchange address, formatted in a manner that is protocol compliant. My application is presumably aware of instrument ID and side and knows how to communicate with the OS in a way that the OS requires in order to render its service in this transaction, but the order message that goes down the wire needn't contain that information - by prior agreement.
Re: Request for Comment: Lightweight Order Message
Jim Kaye / Bank of America Merrill Lynch 28 Jan 2011 1:28AM ET
You can't lose side either. It's tempting to think of + as buys and - as sells, but what about short sells?
Re: Easily processed message - no overhead in accessin message content
Rolf Andersson / Pantor Engineering 27 Jan 2011 11:54PM ET
I managed to be too terse once again. I was referring to the two length/offset fields, not the rest of the data fields. For example:
Re: Easily processed message - no overhead in accessin message content
Mahesh Kumaraguru 27 Jan 2011 6:24PM ET
Fixed Offset(s) would make the message structure inflexible because
Re: Easily processed message - no overhead in accessin message content
Rolf Andersson / Pantor Engineering 27 Jan 2011 5:57PM ET
I was a bit terse in my previous post: the 'tag=value' was meant to be interpreted as _any_ tag/value representation. The way I see it, there is little reason for tagging the two lengths/offsets. If we implement them, they should be part of the message _structure_, probably residing at a fixed offset from the beginning of the message.
Re: Easily processed message - no overhead in accessin message content
Mahesh Kumaraguru 27 Jan 2011 5:49PM ET
I proposed three new tags with the goal of splitting a message into Header, Body and Trailer only. Adding tags to locate sub-parts of the message beyond a certain coarseness of granularity would not work.
Re: Request for Comment: Lightweight Order Message
Philip Beevers / Fidessa 27 Jan 2011 5:11PM ET
So just to be clear, if the exchange is using TCP/IP, the destination address and port should uniquely identify the stock? And in your earlier mobile phone example, the stock is uniquely identified by the phone number I SMS the instruction to?
Re: Request for Comment: Lightweight Order Message
Anders Furuhed / Pantor Engineering 27 Jan 2011 5:11PM ET
You violate protocol layers, adding cost that is perhaps measured in billions per year, without adding any significant benefits processing wise.
Re: Request for Comment: Lightweight Order Message
Rolf Andersson / Pantor Engineering 27 Jan 2011 5:02PM ET
John, this doesn't make sense to me. How would the instrument id and side be conveyed at "a lower communication layer"? Why would you not want to include the instrument id and side in the message?
Request for Comment: Lightweight Order Message
John Harris / BondMart Technologies, Inc. 27 Jan 2011 4:49PM ET
I will posit that it is possible for us to create an order message with just three business-application-level fields:
Re: New Program Trading message types for batching
John Harris / BondMart Technologies, Inc. 26 Jan 2011 7:48PM ET
Thank you for the link, but respectfully, your explanation promises more than the excerpt delivered. Admittedly, I saw only an excerpt, but the authors posit something (the "order-driven market") that, as they define it, does not exist. Economists and other market theorists frequently (and justifiably) resort to hypothetical constructs, whether for investigative or explanatory purposes, and that is what the authors have done with their order-driven market. They do analogize, in the case of call-auction variants of such markets, to real-world examples. They do not provide a real-world example of the continuous-trading flavor. That doesn't disprove their theory, but I nonetheless believe they are guilty of setting up a straw man on specious legs, only to knock him down in favor of their fair-haired boy - the "quote-driven (or "intermediated") market."
Re: Working Group Name
John Harris / BondMart Technologies, Inc. 26 Jan 2011 2:58PM ET
HiPPo also connotes thick skin, which participants in this group will undoubtedly need :-)
Re: Working Group Name
Chris Dunn / RTS Realtime Systems 26 Jan 2011 2:26PM ET
In addition to the following, one must also look at the natural abbreviations that will be associated to the new name.
Re: Working Group Name
John Harris / BondMart Technologies, Inc. 26 Jan 2011 2:02PM ET
Several observations:
Re: New Program Trading message types for batching
Jim Northey / The LaSalle Technology Group 26 Jan 2011 11:28AM ET
Please refer to Equity Markets in Action for a discussion on the differences between quote driven and order driven markets. http://tinyurl.com/5vlwuk5 for an understanding of the differences between quotes and orders. While tradeable quotes can be deemed a subclass or a type of order, I remain unconvinced that the distinction should be lost at the API level.
Re: List of "FAST features because of which FIX over FAST is not suitable for HFT"
Jim Northey / The LaSalle Technology Group 26 Jan 2011 11:24AM ET
Jacob and I discussed this - the use of FAST with possibly only Constant and Default operators should be more than sufficient to meet the needs for high volume trading.
Re: Market structure and the new protocol
John Harris / BondMart Technologies, Inc. 26 Jan 2011 10:08AM ET
To my knowledge, there is no consensus view (at least as yet) of what the new protocol should be, how it should differ from existing FIX, etc. I think it is fair to say, however, that many people - whether purely from their experiences with existing FIX or observation of inefficiencies or capability gaps when applying FIX within certain market segments - think a "clean sheet" protocol-development effort is timely and in order. I would guess that some may support at present only what they would posit as relatively modest, incremental improvements, e.g., binary instead of text. Others already prefer more radical departures or have grander visions.
Re: New Program Trading message types for batching
John Harris / BondMart Technologies, Inc. 26 Jan 2011 9:20AM ET
Thank you, Hanno - terrific post. Please see my comments in-line.
Re: Working Group Name
Nathan Kidd / NMK Consulting Ltd 26 Jan 2011 6:01AM ET
From the various meetings, and mails, it seems we are heading towards something similar to FAST.
Re: Market structure and the new protocol
Hanno Klein / Deutsche Börse Systems 26 Jan 2011 5:42AM ET
your description gives a good overview of many of the fundamentals in trading. Looking at your recommendations, I did not see any gaps related to what the current FIX protocol has to offer. Is the idea to limit the FIX protocol to a well-defined subset of its current capabilities for the purpose of high performance trading environments?
Re: New Program Trading message types for batching
Hanno Klein / Deutsche Börse Systems 26 Jan 2011 5:29AM ET
I agree with the basic notion that tradable quotes are no different from orders and it makes sense to take a step back and analyse what is needed. Quotes are needed as a concept for private quote negotiations where the indicative character dominates. That is also the reason why you find quote messages in FIX in the section "Pre-Trade" and order messages in the section "Trade".
Market structure and the new protocol
John Harris / BondMart Technologies, Inc. 25 Jan 2011 9:34AM ET
During our last call I promised (or threatened :-)) to post some thoughts on how considerations of market structure might influence development of the new protocol. Please see these thoughts below. Recommendations are denoted with three asterisks ("***"). Comments would be most appreciated.
Re: "B 500 AAPL 326.72" a.k.a. Let's Do Something Insanely Useful
John Harris / BondMart Technologies, Inc. 23 Jan 2011 3:05PM ET
Sorry for not being more clear on this subject. My primary point of advocacy is "advance radically the simplicity, convenience, speed, reliability, and accessibility of trading for everyone." If we do our jobs properly, a mobile application developer will be able to create an interaction paradigm as simple as that provided in my example (using Sanskrit if he sees fit). By thinking through what would be required of us as protocol developers in order to enable such application development, we are more certain of producing an insanely useful protocol.
Re: "B 500 AAPL 326.72" a.k.a. Let's Do Something Insanely Useful
Rolf Andersson / Pantor Engineering 23 Jan 2011 1:00PM ET
In my experience, a syntax is difficult to design in a way that it's both simple to use and at the same time flexible and extensible.
Re: List of "FAST features because of which FIX over FAST is not suitable for HFT"
Rolf Andersson / Pantor Engineering 23 Jan 2011 12:39PM ET
good idea. Georges and I have already exchanged a few mails privately re FAST encoding of trading message flows.
List of "FAST features because of which FIX over FAST is not suitable for HFT"
Mahesh Kumaraguru 23 Jan 2011 12:19PM ET
Would it benefit this working group's goals to study why FIX over FAST is not suitable for HFT? MDOWG's work made Market Data work well over FAST, but FIX Trading data / workflows do NOT work well . Maybe we could compile a list of "FAST features because of which FIX over FAST is not suitable for HFT" or is such a list already available somewhere?
Re: Working Group Name
Rolf Andersson / Pantor Engineering 22 Jan 2011 11:55AM ET
I'm open to a name change, although I find "New Protocols working group" a bit too generic for my taste, esp the plural "protocols". I would like us to develop one scalable protocol so that adopters can re-use specs, designs and implementations to as high degree possible.
Re: Easily processed message - no overhead in accessin message content
Rolf Andersson / Pantor Engineering 22 Jan 2011 11:52AM ET
Separation of header and body, possibly different parts of the header and body as well, is in scope.
"B 500 AAPL 326.72" a.k.a. Let's Do Something Insanely Useful
John Harris / BondMart Technologies, Inc. 22 Jan 2011 10:22AM ET
The technology exists for us to advance radically the simplicity, convenience, speed, reliability, and accessibility of trading for everyone. With this new protocol effort, let's be truly ambitious:
Re: Working Group Name
John Harris / BondMart Technologies, Inc. 22 Jan 2011 9:24AM ET
Thank you for the feedback, Mahesh. COUCH trading from the couch? That has a definite appeal :-)
Re: Easily processed message - no overhead in accessin message content
Peter Franzen / Kanonkod 22 Jan 2011 5:37AM ET
> If we add three tags
Easily processed message - no overhead in accessin message content
Mahesh Kumaraguru 22 Jan 2011 4:39AM ET
Page 6 of strawman document says "Easily processed message no overhead in accessing message content". By this, am I correct in assuming that the decoder is able to easily seperate the header, body and trailer ?
Re: Food for thought
Mahesh Kumaraguru 22 Jan 2011 2:03AM ET
I analyzed the 200 byte NewOrderSingle FIX Tag=Value^ message used in your presentation:-
Re: Working Group Name
Mahesh Kumaraguru 22 Jan 2011 1:37AM ET
I agree with you and support your suggestion. After posting http://fixprotocol.org/discuss/read/4fe44b17 I emailed FPL@FIXPROTOCOL.ORG asking them to create a new working group called "Transmission encoding and decoding working group" so that the HFT working group could concentrate on optimizing FIX Semantics / Trading workflows, but the response I got indicated an unwillingness to create an additional working group because it was felt that this two optimizations could be done together better.
Re: New Program Trading message types for batching
John Harris / BondMart Technologies, Inc. 20 Jan 2011 12:19PM ET
Thank you, Don. Let me respond in order to each of your well-taken points:
Re: New Program Trading message types for batching
Donald Mendelson / CME Group 20 Jan 2011 11:23AM ET
There still is a distinction between orders and quotes in many markets. First, quoting is usually associated with a market maker role, and market makers are often obligated to enter two-sided quotes. Second, a market maker typically makes only one public quote per instrument and quotes are implicitly cancel/replaced by market maker / instrument / side combination.
Working Group Name
John Harris / BondMart Technologies, Inc. 18 Jan 2011 1:50PM ET
Let me renew my suggestion, initially made on the working group's first call, that the name of the working group be changed. Continuation of the current moniker will do a disservice to the effort of creating a more efficient protocol, on multiple fronts.
Re: New Program Trading message types for batching
John Harris / BondMart Technologies, Inc. 18 Jan 2011 10:54AM ET
Good discussion - several thoughts in response:
Re: New Program Trading message types for batching
Hanno Klein / Deutsche Börse Systems 18 Jan 2011 9:50AM ET
I agree to some extent. I would not call it "batching" of ERs because only executions of a single order in a single transaction qualify for the . It is still a single order. The same would apply to multiple modifications of the SAME order. I doubt that there is a use case for this. It would rather be a new MassOrderCancelReplace message what you are aiming at. This could have a repeating group of orders. This is similar to a MassQuote where many series can be modified at the same time. We will be looking to add a QuoteExecutionReport to FIX to avoid having to send individual ERs for every quote that gets executed. The key is that <Instrument> needs to be part of a repeating group and the standard ER has it on the root level. We will not change the structure of the standard ER, i.e. only a new message could cover that. More generically, it would be a MassExecutionReport if you want to apply the concept for orders and quotes. We only need it for quotes as this is a single matching event. For orders it would delay the receipt of an ER if the matching engine would wait for more than one order from one and the same submitter until sending him information back. I believe it needs to be aligned with the transaction boundaries of the matching engine, i.e. MassQuote is a single transaction whereas orders (still) hit the engine one by one.
Re: New Program Trading message types for batching
Donald Mendelson / CME Group 18 Jan 2011 9:24AM ET
It is quite common for high frequency traders to enter, modify or cancel multiple orders, all triggered by the same market data event. It could be more efficient to encode, route and deliver those requests in a batch as opposed to multiple individual messages. For one thing, duplication of party identifiers could be eliminated or reduced. For another, at a network level, the latency of routing and delivering the data in one packet (assuming a very efficient encoding) could be less than routing multiple packets. Furthermore, acknowledgement by the receiver could be done at a batch level as well.
Re: New Program Trading message types for batching
Hanno Klein / Deutsche Börse Systems 18 Jan 2011 7:41AM ET
I am not sure about this approach. It blurs the lines between application and session level. NewOrderList is not a batching facility for arbitrary orders. All orders in such a list are linked to each other in a business sense. For example, a basket order for an index consisting of one order per index constituent. Contingent orders are another example where you have only 2 orders (e.g. one cancel the other). The concept of List orders is to enter them in bulk and then to use ListExecute, ListStatus etc. to work on the entire list. Order modifications are done on individual orders with the standard OrderCancelReplace which carries tag 66 ListID to optionally reference a list.
New Program Trading message types for batching
Enio Perpetuo / BM&FBovespa 18 Jan 2011 7:06AM ET
The current NewOrderList (35=E) message doesn’t support modifies.
Re: Subset of FIX Application message types to be considered for HFT
Rolf Andersson / Pantor Engineering 16 Jan 2011 2:45AM ET
I'm editing the scope doc as we speak.
Subset of FIX Application message types to be considered for HFT
Mahesh Kumaraguru 16 Jan 2011 2:30AM ET
A High Frequency Trading protocol is expected to have two parts:-
Re: Pessimistic messaging model for HFT / New message type to Ack receipt of Application messages
Rolf Andersson / Pantor Engineering 16 Jan 2011 12:58AM ET
In my view, the proposed solution will have limited value for a number of error situations while adding message processing overhead.
Pessimistic messaging model for HFT / New message type to Ack receipt of Application messages
Mahesh Kumaraguru 16 Jan 2011 12:19AM ET
FIX Tag=Value^ over TCP/IP Session uses optimistic messaging model "normal delivery of data is assumed (i.e. no acknowledgment of individual messages) with errors in delivery identified by message sequence number gaps."
Re: Underlying protocol - UDP versus TCP
Rolf Andersson / Pantor Engineering 13 Jan 2011 10:57PM ET
The reliability of TCP is over-rated from an application standpoint. TCP is only reliable in the sense that anything that is delivered within a TCP (transport) session is guranteed to be complete, non-duplicated, and in sequence. This is not enough from a transaction point of view. The FIX session protocol is used to complement the transport protocol. Please refer to the discussion in:
Underlying protocol - UDP versus TCP
Mahesh Kumaraguru 13 Jan 2011 10:45PM ET
The scope of HFTWG states "Underlying protocols - don't assume IP but state nature of service required".
Re: Business level functionality for consideration: Ability to replicate transaction information across multiple sites
Hanno Klein / Deutsche Börse Systems 13 Jan 2011 12:08PM ET
I believe this is covered in FIX through Application Sequencing. You can subscribe to ApplID values from the multiple sites. The sender keeps track of the subscribers and sends a single message to multiple sites. The concept includes retransmission requests and more.
Re: Business level functionality for consideration: multiple execution report type info within a single message
Hanno Klein / Deutsche Börse Systems 13 Jan 2011 12:05PM ET
This is covered in FIX since 5.0 SP1 with the addition to the Execution Report. This has been made even more flexible with the new <OrdEventsGrp> which is part of the ISE Order Handling proposal (currently in puiblic review until Jan 28).
Re: Business level functionality for consideration: ability to define message timeout for latency sensitive messages
Hanno Klein / Deutsche Börse Systems 13 Jan 2011 12:00PM ET
Correct EP100 added valid value A="Good For Time" to 59 TimeInForce and tag 1629 ExposureDuration. However, it does not solve the problem of clock syncs and the requirement to have the clock start ticking at the submitter's site. From EP100:
Re: Business level functionality for consideration: multiple execution report type info within a single message
Donald Mendelson / CME Group 13 Jan 2011 9:22AM ET
Yes, match engines may generate multiple execution reports from a single event, e.g. one large order trades with multiple counter parties at different prices. There's no question that it would be more efficient to deliver a bundle of execution reports to an end point as opposed to routing them individually. Not only would the work to encode duplicate party identifiers be reduced, but the routing process would be performed once instead of multiple times.
Re: Business level functionality for consideration: multiple execution report type info within a single message
Georges Gomes 13 Jan 2011 3:12AM ET
This is quite interesting if the exchange manage the "bundling" during the matching. Otherwise, if a software layer in the middle needs to package multi messages together it may be expensive and slow.
Re: Business level functionality for consideration: Ability to replicate transaction information across multiple sites
Rolf Andersson / Pantor Engineering 12 Jan 2011 12:20PM ET
I understand. That is more of a transport issue.
Re: Business level functionality for consideration: multiple execution report type info within a single message
Nathan Kidd / NMK Consulting Ltd 12 Jan 2011 12:19PM ET
In general quite happy with what you describe being part of the protocol layer - works well in FAST...
Re: Business level functionality for consideration: Ability to replicate transaction information across multiple sites
Vitali Vinokour / Lime Brokerage 12 Jan 2011 12:19PM ET
It is a valid concern but I am not sure that it should be addressed at the protocol design level. FIX Drop uses FIX protocol as is; I see no reason why HFT FIX drop copy can't follow the same model.
Re: Business level functionality for consideration: Ability to replicate transaction information across multiple sites
Nathan Kidd / NMK Consulting Ltd 12 Jan 2011 12:04PM ET
One thing could be the ability to replicate data to multiple sites more efficiently.
Re: Business level functionality for consideration: ability to define message timeout for latency sensitive messages
Vitali Vinokour / Lime Brokerage 12 Jan 2011 11:46AM ET
It seems that TIF functionality as it exists now already covers delivery delay. If the order originator wants an order to be valid for 5 minutes, why would the intent change if the order took more than 15 milliseconds to reach the matching engine? I am not sure I see the business case. But if you think that there should be a "start" limit as well as a "stop" limit, it is a matter of adding another timestamp to a TIF order. This is somewhat similar to stop limit orders.
Re: Business level functionality for consideration: multiple execution report type info within a single message
Rolf Andersson / Pantor Engineering 12 Jan 2011 11:42AM ET
just to understand you proposal more in detail; would you be OK with a message encoding that would de-duplicate (remove) the repeated info before sending and re-insert it when the message is received at the other end? Or, do you think that this would still carry too much processing overhead at the receiving end?
Re: Business level functionality for consideration: Ability to pre-send a template message to the remote party
Rolf Andersson / Pantor Engineering 12 Jan 2011 11:38AM ET
http://fixprotocol.org/discuss/read/df2cbc3a
Re: Business level functionality for consideration: Ability to replicate transaction information across multiple sites
Rolf Andersson / Pantor Engineering 12 Jan 2011 11:35AM ET
In what way would efficient drop copies be different from efficient messages in general?
Re: New message types in FIX for communicating Application Objects to aid in recovering broken FIX sessions / failures
Vitali Vinokour / Lime Brokerage 12 Jan 2011 11:31AM ET
I concur. A rich re-send/recovery logic appears to be at cross-purposes with an efficient lightweight protocol design. Transport and /or session layer are responsible for reliable delivery, after that it's an application's responsibility to preserve data properly.
Re: Business level functionality for consideration: ability to define message timeout for latency sensitive messages
Nathan Kidd / NMK Consulting Ltd 12 Jan 2011 11:06AM ET
The problem I see (which seems quite relevant to this group) is that you cannot know when sending a message (especially via TCP) precisely how long it will take for that message to reach its ultimate destination.
Re: Business level functionality for consideration: ability to define message timeout for latency sensitive messages
John Harris / BondMart Technologies, Inc. 12 Jan 2011 9:54AM ET
I agree that enabling the message sender to determine the TIF of an order would be nice, but am doubtful that such a regime is feasible (granting in advance that I am no expert on the subject of clock synchronization).
Business level functionality for consideration: Ability to replicate transaction information across multiple sites
Nathan Kidd / NMK Consulting Ltd 12 Jan 2011 9:49AM ET
For example, perhaps you have 3 co-location exchange installations, but need to replicate transaction information between the sites : perhaps a high frequency mechanism for drop copy style reporting should be included in scope?
Business level functionality for consideration: Ability to pre-send a template message to the remote party
Nathan Kidd / NMK Consulting Ltd 12 Jan 2011 9:47AM ET
For example, during session initialisation, a template message and ID could be sent with pre-defined details of an order, including account allocation, trader id, instrument selection, and maybe even qty, price, etc.
Business level functionality for consideration: multiple execution report type info within a single message
Nathan Kidd / NMK Consulting Ltd 12 Jan 2011 9:40AM ET
Allowing the client application, or exchange (for example) to specify multiple orders or multiple execution reports within a single message / data packet.
Business level functionality for consideration: ability to define message timeout for latency sensitive messages
Nathan Kidd / NMK Consulting Ltd 12 Jan 2011 9:33AM ET
For example, we might want to allow a client application to specify on an order message, that if the message does not reach the matching engine within 15ms of the sending time, the matching engine should discard the message.
Re: New message types in FIX for communicating Application Objects to aid in recovering broken FIX sessions / failures
Anders Furuhed / Pantor Engineering 12 Jan 2011 12:56AM ET
The information flow in a trading setting is typically assymetric where the traditional FIX session layer takes a symmetric view on the message flow.
New message types in FIX for communicating Application Objects to aid in recovering broken FIX sessions / failures
Mahesh Kumaraguru 11 Jan 2011 6:20PM ET
When a FIX Engine / application is recovering from a crash / failure, it restores its internal state using persistent data to the last known best state after the last persistence record was written and before the failure happened. After having restored itself, if its wants to validate its state using data from the counterparty(ies), it can do a resend request 0 to infinity and reconstruct the state of Application objects using the data contained in the responses to resend requests (not a network friendly solution but works ;-). Resend request processing is synchronization at the FIX Engine level, these new set of messages to aid connected counterparty applications synchronize themselves operates at Application level. In resend request processing logic, there is no way for an application to ask its counterparty what messages it sent to counterparty. This sounds confusing - this is like me asking you in the middle of our conversation "sorry, What was I telling you ?" or "friend, what were we talking about ?". I propose that the following new message types be created in FIXProtocol / HFT to help communicate Application Objects (Non-Session objects).
Re: Legal framework
John Harris / BondMart Technologies, Inc. 11 Jan 2011 9:07AM ET
> John, relevant questions and not only for HFTwg but in general. I'll check with the FPL Program Office and provide an update at the next meeting (not this afternoon).
Re: Legal framework
John Harris / BondMart Technologies, Inc. 11 Jan 2011 9:07AM ET
The FPL website and various materials published there do address intellectual property. FPL claims the copyrights, for example, to FIX specifications.
Re: Flexibility versus performance
Rolf Andersson / Pantor Engineering 11 Jan 2011 8:25AM ET
Taking a somewhat different perspective, I would argue that we need to _increase_ the flexibility as compared to FIX. This is possible by allowing increased configurability, re-using the good parts of FIX and relaxing some of the current limitations.
Re: Flexibility versus performance
Mark Reece / HSBC 11 Jan 2011 7:34AM ET
I agree that the business flexibility of FIX must be retained. Indeed, a lot of the FIX IPR is captured in these specifications which describe some very varied trading models.
Re: Legal framework
Rolf Andersson / Pantor Engineering 11 Jan 2011 5:05AM ET
John, relevant questions and not only for HFTwg but in general. I'll check with the FPL Program Office and provide an update at the next meeting (not this afternoon).
Re: Legal framework
Mahesh Kumaraguru 11 Jan 2011 1:57AM ET
Are these not already covered for all the work available in the FIXProtocol website / under FPL ? Would not the outputs of HFT working group also fall under the same terms of license ?
Flexibility versus performance
Mahesh Kumaraguru 11 Jan 2011 12:55AM ET
I agree with you regarding preserving the FIX specification because there is a lot of business value into it. One thing I would like to preserve in any HFT protocol design is the flexiblity FIX has to offer. For example, OUCH protocol of NASDAQ is faster but not flexible - the only thing that can be done using OUCH is trade on NASDAQ. But FIX can be used to trade across most geographies of the world on most asset classes using many diverse trading styles. FIX TCP Sessions can transport non FIX data reliably. FIXProtocol is much more than just a trading protocol.
Legal framework
John Harris / BondMart Technologies, Inc. 10 Jan 2011 3:42PM ET
All,
Re: Two seperate battles - Optimizing Transmission and FIX Semantics
Mark Reece / HSBC 5 Jan 2011 5:21AM ET
Rolf, I agree with the testing concerns. We should develop a methodology for testing and proving performance. It might start with off-the-wire capture of inbound and outbound message pairs to determine the latency with an agreed application which takes the incoming message and builds the outgoing message.
Re: Two seperate battles - Optimizing Transmission and FIX Semantics
Rolf Andersson / Pantor Engineering 5 Jan 2011 4:48AM ET
Mahesh, I fully agree that separation of concerns is important. I see two reasons for this:
Re: Two seperate battles - Optimizing Transmission and FIX Semantics
Philip Beevers / Fidessa 5 Jan 2011 4:32AM ET
This is a suggestion I made privately following the first meeting; I think it's a good idea. I actually suggested three streams:
Re: Encoding Strings in compact binary form
Georges Gomes 5 Jan 2011 4:26AM ET
I take that for next update of my document.
Re: Two seperate battles - Optimizing Transmission and FIX Semantics
Georges Gomes 5 Jan 2011 4:21AM ET
I like this.
Two seperate battles - Optimizing Transmission and FIX Semantics
Mahesh Kumaraguru 5 Jan 2011 4:16AM ET
We could seperate the work of High Frequency Trading working group into two parts :-
Re: Food for thought
Georges Gomes 5 Jan 2011 4:16AM ET
Prepared messages are all about defining them on the fly when you decide it. HFT boxes could prepare replaces of a specific order if required.
Re: Encoding Strings in compact binary form
Mahesh Kumaraguru 5 Jan 2011 4:04AM ET
When the String is longer than 63 bytes, then the length of the String in the field header is set to 000000 bits and the first byte(s) of the String contains the length of the String. I can think of two approaches here
Re: Food for thought
Georges Gomes 5 Jan 2011 3:57AM ET
My main concern was to preserve the FIX specifications. Because there is a lot of business value into it. Redesigning it would be a waist of time in my perspective. But maybe I sticked too much with the tag=value style.
Re: Encoding Strings in compact binary form
Rolf Andersson / Pantor Engineering 5 Jan 2011 3:44AM ET
other (similar) encodings have multiple string types; in addition to the 0-63 byte string type, you could also define one or more types that can encode strings with >63 bytes. For example, you could assign a type that has a 4 byte length followed by the data:
Re: Encoding Strings in compact binary form
Georges Gomes 5 Jan 2011 3:08AM ET
I initialy thought of encoding string with an additional varint in front of the string to specify the size of the string. varint can bring up to 128 char len.
Re: Performance baselining
Philip Beevers / Fidessa 4 Jan 2011 9:04AM ET
My thoughts were that we would get hold of some real production traffic for a typical session with a FIX-based MTF. We'd then build a test harness which takes the place of the MTF, and, given suitable orders/amends/cancels, would build corresponding acknowledgements, executions etc.
Re: Food for thought
Philip Beevers / Fidessa 4 Jan 2011 6:59AM ET
Hi Georges - at a high level this all looks good to me, i.e. I'm very much in favour of trying out a better encoding of FIX semantics to begin with. A few points if I may - apologies if you covered these areas and I missed them:
Re: Session protocol for HFT / Underlying Transport protocols ? Transport independence framework ?
Rolf Andersson / Pantor Engineering 2 Jan 2011 12:08PM ET
not sure I understand what you are suggestion here, but let me try to explain how I interpret this aspect of the wg scope.
Session protocol for HFT / Underlying Transport protocols ? Transport independence framework ?
Mahesh Kumaraguru 2 Jan 2011 9:16AM ET
Page 4 of HFT Intro v02.ppt under "Scope of working group" states
Re: Byte versus bit optimization for HFT / FIX over FAST //
Rolf Andersson / Pantor Engineering 2 Jan 2011 1:30AM ET
these are important questions. I think the answers depend on how we want to trade off compactness, speed, processing overhead, implementation complexity (and possibly other factors).
Byte versus bit optimization for HFT / FIX over FAST //
Mahesh Kumaraguru 2 Jan 2011 1:06AM ET
Since FAST is a byte oriented format (as is BMF and OPF), should HFT try the bit oriented format because any further work on byte oriented optimizations could be carried out under FAST improvements.
Need for BodyLength on FIX messages over TCP Sockets
Mahesh Kumaraguru 30 Dec 2010 6:46PM ET
TCP Socket based communications allows only to send packets of data between applications. When a Java application opens a socket and starts reading the input stream, how can the application know how much to read for the current FIX message ? If both BodyLength and Checksum are removed, then how to breakup a stream of bytes into a series of messages? In message oriented middleware, the middleware delivers seperate messages to the application and hence a message BodyLength serves no useful purpose.
Re: Food for thought
Mark Sipos / FIX Flyer LLC 30 Dec 2010 5:17PM ET
Also it is worth remembering that in the dark ages of FIX (pre mass Internet circa '93) TCP/IP was not a given. There was work with X.25 if I'm not mistaken - Here's looking at you, BLAM!
Re: Food for thought
Donald Mendelson / CME Group 30 Dec 2010 4:58PM ET
Alex,
Re: Food for thought
alex o / self 30 Dec 2010 4:20PM ET
finally somebody stated the obvios - the message length and check sum are arguably the worst/completely unnecessary "features" of Fix protocol for financial communications. First is just plain stupid - reading from a socket and parsing it byte by byte to figure out a length , while excluding the end of the message - only a business type person could come up with something moronic like that. Second is completely unnecessary for communications going over private networks over TCP(!!!!!!) protocol.
Re: Encoding Strings in compact binary form
Rolf Andersson / Pantor Engineering 28 Dec 2010 5:42PM ET
this is reminiscent of one of the early mdowg proposals at the end of January 2005. What we tried to create at the time was a sort of virtual machine instruction set that when executed generated unpacked data. The proposal was called "OPF - Opcode Binary Format" and may be available in some archive. The proposal was later superceeded by "BMF - Byte Mapped Format" that later in an enhanced form became FAST.
Re: Food for thought
Rolf Andersson / Pantor Engineering 28 Dec 2010 5:09PM ET
your point on looking at more radical changes is well made. Whereas the syntactical level (encoding and decoding) is a necessary step I believe that the dramatic improvements will be found elsewhere.
Performance baselining
Rolf Andersson / Pantor Engineering 28 Dec 2010 5:01PM ET
All,
Re: Food for thought
Vitali Vinokour / Lime Brokerage 28 Dec 2010 11:44AM ET
This is great analysis. There are many useful ideas here on how to compress FIX messages and reduce processing time. What is not quite clear to me is which parts of FIX you are suggesting to keep and why. It appears that you aim to achieve performance gains by addressing message encoding while preserving FIX semantics in full, both session level and application level. I concede that simply switching to binary message format is perhaps the best way to improve performance. However, given that FIX engines will need to be rewritten to process new message formats anyway, it is relevant to ask whether more radical changes to FIX message structure and semantics is worth considering to attain even better performance.
Encoding Strings in compact binary form
Mahesh Kumaraguru 26 Dec 2010 3:10PM ET
I read thru the section "Encode Fields in compact binary form" and wanted to get a clarification. On pages 25/48 and 26/48, for all formats other than String, the length of the value in bytes is known from the value of the "TBD" 7 bits. But in case of String, the parser will know its a String value, but would not know the length of the String and hence where next Field header starts.
Food for thought
Georges Gomes 23 Dec 2010 2:21AM ET
Hi All,
Welcome
Rolf Andersson / Pantor Engineering 21 Dec 2010 4:24PM ET
All, welcome to the HFT forum at fixprotocol.org
- - - - - - - - - - - - - - - - - - - - - - - - -
[End of Search results]
- - - - - - - - - - - - - - - - - - - - - - - - -
Now if a button with label "Show recent activity" is added next to the search button and when user clicks this new button, internally send "a search for any of "a e i o u"" to server and display the results.
This search works only within each of the individual discussion forums, but does not work across discussion forums.
Regards,
K. Mahesh
@FIXProtocol
