MiMo Pro runs on Jetson AGX Thor at 0.646 tokens/sec, loading a ~566GB FP8 model from disk and using 128GB memory; Dario notes lack of model transparency
Read original on r/LocalLLaMA ↗Tracked my first run of MiMo Pro on a Jetson AGX Thor at a scorching 0.646 tokens/sec,.. but it DID technically load and run, so it counts. Every decoder layer, routed expert, SSD read (..a good bit more) traced and...
Original headline: "Dario: “With open source you can see the source code. Here we can’t see inside the model.” My 0.646 t/s MiMo-2.6-Pro crammed thru 128GB of Jetson AGX Thor: “Yeah, well, that’s just, like, your opinion, man.”"