Late data on a Massive WebSocket comes from one of two places, and from inside your application they look the same: your client reading the stream more slowly than it arrives, or events that were already late when they reached your machine. Rewriting your client fixes only the first, so tell them apart before you change anything. One property of the connection settles a common case on its own: every channel on a connection arrives on one ordered stream, so a client that reads the socket too slowly delays every channel equally and cannot make one channel late by itself.
Applies to
- Plans: every plan with real-time WebSocket access. If every event is about 15 minutes old, check the URL first: the WebSocket quickstart gives wss://delayed.massive.com as the 15-minute delayed feed and wss://socket.massive.com as the real-time one.
- Feeds: every cluster under wss://socket.massive.com: /stocks, /options, /indices, /futures, /forex and /crypto.
- Asset classes: stocks, options, indices, futures, forex and crypto.
How to measure how late the data is
Measuring how late Massive WebSocket data is takes your own clock, corrected, and the event's t field. The WebSocket documentation gives t as a Unix timestamp in milliseconds on the trades, quotes and index value channels of every cluster. On stocks trades and quotes, t is the time the SIP (Securities Information Processor, the consolidated feed for US stocks) published the event, according to the stocks trades documentation.
- Measure your clock's offset from true time and correct every reading by it. What is the average latency for Massive's WebSockets? shows how, and why a figure taken without the correction is mostly clock error.
- Record the receive time the moment your code reads each frame off the socket, before the frame goes into any queue of your own.
- Subtract t from the corrected receive time. The result is how old the event was when it reached you.
- Measure on trades, quotes or index values, not on per-second or per-minute aggregate bars. Bars are published after their window closes on purpose, as the late-bar article explains.
- Keep the median, the 99th percentile and the maximum per channel and per minute, rather than one average, so that one late channel or one bad minute stands out.
To compare channels later, subscribe to at least two on the same connection, such as trades and quotes for one ticker, and check that each parameter is acknowledged, because a parameter with no acknowledgement has not taken:
{"action":"subscribe","params":"T.SPY,Q.SPY"}
Response
[{"ev":"status","status":"success","message":"subscribed to: T.SPY"},{"ev":"status","status":"success","message":"subscribed to: Q.SPY"}]
On /stocks the acknowledgement carries one status object per parameter, all in one message.
How to tell a slow client from late delivery
Telling a slow client from late delivery takes one check on your own machine, made while the lag is happening: the socket's receive queue. The receive queue is the data the operating system has already received for your connection and your program has not read yet, and the Recv-Q column shows it in bytes, as the netstat manual describes. Find the line for your connection to port 443:
# macOS
netstat -an -p tcp
# Linux
ss -tn
- A receive queue that climbs and keeps holding data means your client is not reading as fast as data arrives. Read frames off the socket into a queue and parse them on another thread, and if that is not enough, subscribe to less, as the WebSocket documentation advises.
- A receive queue that stays near empty while events are still old, with the age taken at the socket read, means nothing on your machine is holding them back. They were already late when they reached it, delayed on the network path or on our side. The network path includes your own link: when a subscription needs more bandwidth than the link has, the data waits before it reaches your machine and the receive queue still stays near empty. So count the bytes your client reads per second over a minute and compare that with your link's capacity before you report it. The WebSocket documentation notes that a wired connection, rather than Wi-Fi or a VPN, can help reduce latency; for anything else, send us the details listed below.
- One channel or one symbol running late while others on the same connection are current is not your socket read. A connection is one TCP stream, which delivers bytes in order (RFC 9293, section 2.2), so a client that reads too slowly holds back every channel behind it equally. That reasoning covers only ages taken at the socket read: a per-channel queue inside your own application can make one channel late on its own.
- If nothing changed on your side, look at massive.com/system for live status and at massive-status.com for posted incidents.
What a client falling behind looks like
A client falling behind looks like this in the receive queue. In this example, after the close, a client on macOS with one /stocks connection subscribed to T.* and Q.* reads normally, stops reading for 15 seconds, then reads again:
| Moment | Receive queue | Event age |
|---|---|---|
| Reading normally | 0 bytes | Baseline |
| 2.5 seconds after reading stopped | 178,360 bytes | Not read yet |
| 15 seconds after reading stopped | 178,360 bytes | 15.2 seconds, for the first event read on resuming |
| About 2 seconds after reading resumed | Under 2 KB | Back to baseline |
The queue fills within 2.5 seconds and then holds exactly the same size until the client reads again, and that shape is the thing to look for. The size itself varies: the same setup can level off at under 200 KB or at over 700 KB.
When options quotes run behind
Options quotes running behind on a Massive WebSocket call for the same check, with one detail specific to the feed. The Options Quotes documentation cites the high bandwidth and message rates of options quotes as the reason it allows a maximum of 1,000 option contracts per connection, and lists real-time Options Quotes on Options Advanced and Options Business + Expansion.
- Subscribe to quotes (Q) and trades (T) on the same /options connection, and measure the age of each separately. t is Unix milliseconds on both.
- If quotes are minutes old while trades on the same connection are current, a slow socket read is not the cause, because both arrive on the same ordered stream and a client that reads too slowly delays both equally.
- Check that you take the age at the socket read rather than after a queue in your application that only quotes pass through, then send us the connection ID and the times.
The cap, and the message a subscribe gets when it would cross it, are in How many tickers can you subscribe to on a single Massive WebSocket connection?
What to send us
What to send us when Massive WebSocket data arrived late into an empty receive queue is what lets us find your connection. Start a chat at massive.com/contact with:
- The connection ID of the connection that ran behind, from the connections tab of your WebSocket dashboard, not the ID of one you opened afterwards.
- The window, start and end, in UTC.
- The cluster, the channels and roughly how many tickers you subscribe to.
- Your age percentiles per channel and per minute for that window.
- Your receive-queue samples from the same window.
Why this happens
Late Massive WebSocket data happens in one of two places, and only one of them is on your machine. The WebSocket documentation asks clients to process messages quickly to avoid server-side buffering, and says the server may disconnect a client it detects consuming slowly. During high-volume periods the server bundles multiple events into one message, so the amount of work in each read is uneven and a client that keeps up on average can still fall behind in a burst. When your client does keep up and the data is still late, the delay happened before the data reached your machine. The age alone cannot tell you which of the two you have, and the receive queue can.
If you see an error
Lag that builds until the connection drops is covered in Why do Massive WebSocket disconnects happen?, including how to reconnect so the stream resumes.
A per-second bar that arrives about two seconds after its second is the late-trade window, not lag. Why am I receiving a late aggregate bar through Massive's WebSockets? explains it.
A negative age means your clock is behind, not that the event arrived early. Correct by your measured offset before reading anything else into the numbers.
