home.social

#onnx — Public Fediverse posts

Live and recent posts from across the Fediverse tagged #onnx, aggregated by home.social.

  1. I'm writing a C# app with ONNX for facial detection and in part of it doing `tensor[0, 2, y, x] = r`. After running the app for a day it used almost 8 GB memory. I coudldn't really find any obvious memory leaks, but found that the indexer actually created a temporary `int[]`. I refactored the indexer to `tensor[2 * width * height + y * width + x] = r`, and did some other (unrelated changes). And now a day later it's at 0.4 GB. Is that how this works?

    To pharaphrase the Haskell saying:
    > The [garbage collector] should have protected me from this!

    #csharp #onnx #optimisation

  2. should figure out their strategy because they have:

    1. plugins for GPU and NPU,
    2. plugin for GPU
    3. for PyTorch
    4. intel-npu-acceleration-library for PyTorch
    5. oneDNN neural network math kernels

    And for they have both OpenVino and oneDNN runtimes.

    Best of all I haven't reliably gotten the NPU to work using any permutation of them lol..