Can you use autoregressive diffusion to generate market data?

(blog.janestreet.com)

34 points | by jsomers 13 hours ago

5 comments

  • stult 1 hour ago
    There is no model of the market that can remain stably accurate because the market will inevitably incorporate the insights of any model that is accurate until those insights are no longer accurate
    • teravor 34 minutes ago
      in order for your model to accomplish that, you would get very rich.

      there will also likely always be more. in the limit in order to get an edge your model would start to infer insider information. for example, it's common knowledge by now that satellite imagery is used to measure car numbers in parking lots, that's a proxy for insider information.

      so it's not even so much the model as it is the data.

      even being able to forecast weather better than publicly available methods can be leveraged to gain a significant edge.

    • majormajor 26 minutes ago
      This assumes that any accurate model will inevitably get big enough to be noticable by the rest of the actors.
    • tylerflick 59 minutes ago
      AKA the efficient markets hypotheses.
      • zdc1 36 minutes ago
        Or, thankfully, for Jane Street: the (eventually) efficient market hypothesis

        There's definitely alpha out there, but I wouldn't want to make it my job to look for it

  • armcat 40 minutes ago
    The real story here is this wonderful exposition in applying diffusion models to a time series data that is neither discrete nor continuous. It’s always fascinating to see diffusion models applied in different scenarios, same with diffusion language models.
    • pottertheotter 7 minutes ago
      I think you’re the only other person that read the article.
  • dzink 1 hour ago
    The market has modes and reverts behavior when it switches them. Thus happy bouncy becomes hammered stammered. The prediction models fall hook and sinker for that.
  • reedf1 1 hour ago
    No
  • TheOtherHobbes 2 hours ago
    "Past performance is not indicative of future results."
    • socializer 41 minutes ago
      You hear that often, but if you squint your eyes, the entire idea of index funds is just that: they outperformed stock-pickers in the past, so you should put money into them to get higher returns in the future. There's no fundamental index fund investment thesis other than "past performance is indicative of future returns".

      That thesis is at least to some extent self-fulfilling, because there's so much money flowing into index funds that prices of all the underlying assets keep moving up, and there's probably not enough money trying to bid against that / arbitrage the excesses away.

      A similar thing could happen with AI. Markets are efficient only if the world isn't in some sort of a trance.

      • pottertheotter 10 minutes ago
        That's not the idea behind index funds. It's arithmetic. The aggregate return of active investors, before fees, is the market return. Once you subtract fees, it's below the market return. While some active managers' performance less fees is higher than the market return, it's very difficult to predict which will perform this way. So your best bet is to own the market through a broad index fund that has almost no cost.

        If you want to read about this, see Sharpe (1991), The Arithmetic of Active Management.

      • majormajor 23 minutes ago
        I don't think that's a particularly accurate assessment of the idea behind index funds.

        The point of index funds isn't "these outperform all pickers, so they'll outperform all pickers in the future."

        I think the idea is more around a combination of:

        - you'll have much lower risk trying not to pick the right picker (or pick the investments yourself)

        - the median picker is probably not very good (approached in two directions: sizable pickers that hit on an edge will likely be copied until the edge is gone, and smaller pickers are extremely unlikely to have enough specialized info or skills to excel).

      • Leif24 34 minutes ago
        > you should put money into them to get higher returns in the future.

        Higher returns than what? I thought the whole point of buying broad market index funds was to simply get the market returns. For this thesis to make sense, you simply must assume that companies, in aggregate, make money - not that any particular company will follow past performance. If you don't think companies make money, then what are you doing buying equities?