Street Fighter, 2026 - ★★: https://letterboxd.com/bradr3d/film/street-fighter-2026/
Search
20 results for “bradr”
-
The VelociPastor 2, 2026 - ★★★★: https://letterboxd.com/bradr3d/film/the-velocipastor-2/
-
The VelociPastor, 2017 - ★★★★: https://letterboxd.com/bradr3d/film/the-velocipastor/
-
Trump's big midterm problems emerge in small-town debate
By Brad RyanA political debate in Maine offered multiple reminders of times when Donald Trump has dismissed Americans' concerns as either conspiracy theories or stupidity.
-
John Berger, The White Bird (1987)
You can't talk about aesthetics without talking about the principle of hope and the existence of evil.
-
Two old boats, napping like cats in the shade, curled up next to the autumn colored trees.
-
Badrinath #Temple in #Nepal is part of the group of four Char Dham #Temple s, which are the local #Bhaktapur Durbar Square version of the four Char Dham #Temple s in India. We liked its design - have you seen it? https://backpackandsnorkel.com/Nepal/Day4/5412d-BadrinathTemple/
-
Attaques visant le juge Youssef Badr : ouverture à Paris d’une enquête pour cyberharcèlement
🗞️ Le Monde.fr - 🕐 16/10 19:35
Le parquet de Paris a fait savoir à l’Agence France-Presse, vendredi 16 octobre, avoir ouvert une enquête pour retrouver les auteurs d’une « nouvelle vague » de cyberharcèlement visant le juge Youssef Badr, à la suite de la publication d’un numéro du... [2157 chars]
🔗 https://www.lemonde.fr/societe/article/2026/10/16/attaques-visant-le-juge-youssef-badr-ouverture-a-paris-d-une-enquete-pour-cyberharcelement_6795628_3224.html
#actu #news #presse #lemonde.fr -
Pris à partie par " Frontières ", le juge Youssef Badr reçoit le soutien de la magistrature
🗞️ Le HuffPost - 🕐 15/10 21:21
Des soutiens de taille pour contrer une presse d’extrême droite des plus virulentes. Le Conseil supérieur de la magistrature a dénoncé, ce jeudi 15 octobre, un « risque de déstabilisation et de discrédit de l’institution judiciaire » en raison des at... [3020 chars]
🔗 https://www.huffingtonpost.fr/france/article/pris-a-partie-par-frontieres-le-juge-youssef-badr-recoit-le-soutien-de-la-magistrature_340397.html
#actu #news #presse #lehuffpost -
https://www.europesays.com/fr/1246114/ Attaques visant le juge Youssef Badr: ouverture à Paris d’une enquête pour cyberharcèlement : Actualités #actu #ActuParis #Actualités #ActualitésParis #enquête #EU #europe #FR #France #harcèlement #internet #Jugé #médias #News #NewsParis #Paris #ParisNews #presse #RépubliqueFrançaise
-
Rangers fail to overtake Celtic as diabolical defending in Dundee sees top spot slip away – 5 talking points
The 2-2 draw at Dens Park brings the Glasgow giants seven game winning streak to an end Ryan…
#NewsBeep #News #Headlines #BadredineBouanani #DerekMcInnes #DundeeFC #KevinKelsy #KimMin-su #NicolasRaskin #RangersFC #RangersFCTransferNews #ScottishPremiership #StevenPressley #TransferTalk #UK #UnitedKingdom
https://www.newsbeep.com/774888/ -
Rangers fail to overtake Celtic as diabolical defending in Dundee sees top spot slip away – 5 talking points
The 2-2 draw at Dens Park brings the Glasgow giants seven game winning streak to an end Ryan…
#Football #Soccer #News #BadredineBouanani #DerekMcInnes #DundeeFC #KevinKelsy #KimMin-su #NicolasRaskin #RangersFC #RangersFCTransferNews #ScottishPremiership #StevenPressley #TransferTalk
https://www.europesays.com/football/74563/ -
Derek McInnes has his own Rangers player pledge tested as forgotten recruit lands Dundee shot – predicted XI
The Light Blues are looking to return to top spot in the table with a win at Dens…
#Football #Soccer #News #BadredineBouanani #DerekMcInnes #DjeidiGassama #DundeeFC #JamesPenrice #KevinKelsy #KimMin-su #OlwethuMakhanya #Premium #RangersFC #RyanNaderi #ScottishPremiership #TuurRommens
https://www.europesays.com/football/74469/ -
FM, Syrian Finance Minister discuss strengthening bilateral cooperation
أبو بكر السوري Minister of Foreign Affairs, International Cooperation and Egyptian Expatriates Badr Abdelatty had talks with Syrian…
#EuropeSays #Headlines #News #Europe #Europa #EU #Syria #BadrAbdelatty #BilateralCooperation #InternationalMonetaryFund #MinisterofForeignAffairs
https://www.europesays.com/iran/332041/ -
https://www.europesays.com/iran/332041/ FM, Syrian Finance Minister discuss strengthening bilateral cooperation #BadrAbdelatty #BilateralCooperation #InternationalMonetaryFund #MinisterOfForeignAffairs #Syria
-
https://www.europesays.com/africa/476547/ Egypt Rejects Military Access to Red Sea for Landlocked Countries #AsharqBloomberg #BadrAbdelatty #cairo #Egypt #egyptians #Ethiopia #MiddleEast #news #SaudiArabia
-
RE: https://mastodon.social/@bader2000/117456620988364991
Cada día mis hijos y yo sufrimos el hambre y la desesperación en Gaza. Mi marido se encuentra en prisión y me enfrento a esta lucha sola. Mis hijos estan muriendo de hambre y no tengo comida para darles. Me rompe el corazón verles sufrir.
Por favor, no nos dejéis solos con este dolor. Cualquier donación o compartir este post nos puede ayudar a sobrevivir.
Por favor oid nuestro llanto y estad con nosotros.
=====================
Además del hambre también se está acercando el invierno y necesitan ayuda para preparar su tienda:
https://mastodon.social/@bader2000/117388770134997971
=====================
La última donación es de hace 3 días ($25)
https://gaza-verified.org/people/@bade[email protected]/
#Gaza #GazaVerified #Palestine #FediForGaza #Gaza1107Days #Gaza1107th #Donate #GazaDonate #HelpGaza #GazaHelp #HelpBaderHasnFamily #GazaChildren #ChildrenOfGaza #Hunger #Starving #WinterIsComing #WinterInGaza #GazaTent #MutualAid #MutualAidRequest #HumanitaryAid #HumanitarianAid #BoostForHelp #Boost4Help #BoostForReach #Boost4Reach #BoostWelcomed
-
"En juin, le juge était déjà dans le viseur du directeur de la publication de Frontières, Erik Tegnér, aussi chroniqueur sur #CNews. Ce dernier dénonçait sa condamnation à 6 mois de prison avec sursis par le tribunal de Bobigny, présidé par Youssef Badr, pour avoir divulgué des données personnelles d'avocats spécialisés en droit d'immigration. Tegnér avait appelé ses abonnés à se "mobiliser" sur X déclenchant une vague d'insultes et de menaces à l'encontre du magistrat."
-
Gérald Darmanin estime qu’" aucun magistrat ne doit être menacé ou insulté ", en réaction aux attaques contre un juge de Bobigny
🗞️ Le Monde.fr - 🕐 15/10 20:55
Le garde des sceaux, Gérald Darmanin, a apporté, jeudi 15 octobre, son « soutien » à un juge du tribunal de Bobigny, Youssef Badr, « qui fait l’objet d’insultes et de menaces sur les réseaux sociaux », après que le magazine d’extrême droite Frontière... [4708 chars]
🔗 https://www.lemonde.fr/societe/article/2026/10/15/gerald-darmanin-estime-qu-aucun-magistrat-ne-doit-etre-menace-ou-insulte-en-reaction-aux-attaques-contre-un-juge-de-bobigny_6794794_3224.html
#actu #news #presse #lemonde.fr -
Computer Vision with .NET
A deep-dive walkthrough of integrating computer vision into .NET applications — covering image classification (ML.NET and ONNX Runtime), object detection with YOLO-style models, OCR (Azure AI Vision, Document Intelligence, Tesseract), face and object recognition and the legal and ethical constraints around them, image embeddings and visual search, Azure AI Vision, OpenCV integration with OpenCvSharp, image processing APIs (ImageSharp, SkiaSharp and others), multimodal LLMs versus classical vision, and building a production vision service in ASP.NET Core.
Table of Contents
- Introduction
- The Computer Vision Landscape in .NET
- Image Classification
- Object Detection
- OCR
- Face and Object Recognition
- Image Embeddings
- Azure AI Vision
- OpenCV Integration
- Image Processing APIs
- Multimodal LLMs vs. Classical Computer Vision
- Building a Vision Service in ASP.NET Core
- Evaluating Vision Systems
- Choosing the Right Approach
- Common Pitfalls
- Quick Reference Table
- Conclusion
Introduction
Computer vision is the field of getting software to understand images and video: what is in a photo, where each object is, what a document says, whether two pictures show the same thing. For .NET developers it has become practical in two ways at once: managed cloud services (Azure AI Vision and Document Intelligence) turn many tasks into a single API call, and the ONNX ecosystem lets you run state-of-the-art open models inside your own process, on your own hardware, with no Python in production.Image / video frame
↓
Decode + preprocess (ImageSharp / SkiaSharp / OpenCV: resize, normalize, orient)
↓
Model (cloud API | ONNX Runtime | ML.NET | multimodal LLM)
↓
Postprocess (softmax, boxes + non-max suppression, text assembly, thresholds)
↓
Business logic (store, search, flag for review, trigger a workflow)This is the pipeline behind every technique in this guide. Most vision bugs hide in the first and third stages — preprocessing that doesn't match what the model was trained on, and postprocessing that misreads the model's output — not in the model itself. The guide's emphasis follows: how to wire each task correctly, how to choose between a managed API, a self-hosted model, and a multimodal LLM, and how to measure whether the result is good enough.
A note on versions and services. Cloud vision services evolve and retire features (older Computer Vision API versions, Custom Vision, and some Face capabilities have been retired or restricted over time), SDK package names and signatures change between releases, and model output formats differ across model versions. Treat the samples here as illustrating the shape of each integration, compile them against the package versions you install, and check current service documentation for availability, regions, limits, and lifecycle status before committing to a design.
1. The Computer Vision Landscape in .NET
The tasks
CLASSIFICATION What is in this image? "cat", "defective", "invoice" one label (or a few) per image
OBJECT DETECTION What objects are where? boxes + labels + confidence "3 cars at these coordinates"
SEGMENTATION Which pixels belong to which object? per-pixel masks medical imaging, background removal
OCR What text is in this image? text + positions receipts, IDs, signs, scanned PDFs
DOCUMENT ANALYSIS Structure and fields of a document tables, key-value pairs invoices, forms
FACE TASKS Detect, verify, identify faces boxes, embeddings, match scores (heavily regulated — Section 5)
EMBEDDINGS A vector capturing the image's meaning float[ ] visual search, dedupe, recommendations
CAPTIONING / VQA Describe or answer questions about image natural language accessibility, search, assistantsThe four ways to get a vision capability
1. MANAGED CLOUD API Azure AI Vision (Image Analysis, Read OCR), Azure AI Document Intelligence, Azure AI Face.
+ No ML expertise, fast to build, maintained and improved for you.
- Per-call cost, data leaves your boundary (unless containers), limited customization,
vendor features change and retire.2. PRE-TRAINED MODEL ON Download an ONNX model (ResNet, MobileNet, YOLO, CLIP, ArcFace...) and run it in-process
ONNX RUNTIME with Microsoft.ML.OnnxRuntime.
+ Full control, no per-call fee, data stays local, runs offline/at the edge, can use GPU.
- You own preprocessing/postprocessing, model hosting, performance, and licensing checks.3. TRAIN YOUR OWN Fine-tune a model on your labeled images — ML.NET transfer learning for classification, or
train in PyTorch/TensorFlow/Azure ML and export to ONNX.
+ Best accuracy on your specific domain (defect types, your products).
- Needs labeled data, evaluation discipline, and MLOps.4. MULTIMODAL LLM Send the image to a vision-capable chat model (see Section 10).
+ Extremely flexible, zero-shot, handles messy real-world images and open-ended questions.
- Cost, latency, non-determinism, hallucination risk, weaker at precise localization and
counting, and data-privacy considerations.The core .NET packages
Microsoft.ML.OnnxRuntime (+ .Gpu / .DirectML) Run ONNX models in-process (CPU, CUDA, DirectML)
Microsoft.ML + Microsoft.ML.Vision + Microsoft.ML.ImageAnalytics ML.NET image classification via transfer learning
SixLabors.ImageSharp (+ .Drawing) Managed, cross-platform image processing
SkiaSharp Fast 2D graphics and image processing (Skia)
OpenCvSharp4 (+ runtime packages) OpenCV bindings: classic CV, video, DNN
Azure.AI.Vision.ImageAnalysis Azure AI Vision Image Analysis 4.0 (captions, tags, objects, read OCR, people)
Azure.AI.DocumentIntelligence Document OCR, layout, and prebuilt/custom extractors
Tesseract Open-source OCR engine wrapper
Microsoft.Extensions.AI Vision-capable chat models via IChatClient (Section 10)
System.Numerics.Tensors Vector math (cosine similarity) for embeddingsA decision shortcut
"Read the text/fields from documents" -> Azure AI Document Intelligence (or Vision Read for plain OCR)
"Describe / tag / caption general photos" -> Azure AI Vision Image Analysis, or a multimodal LLM
"Detect common objects (people, cars, ...)" -> Azure AI Vision Objects/People, or a YOLO ONNX model
"Classify MY specific categories (defects, SKUs)" -> ML.NET transfer learning or a fine-tuned model exported to ONNX
"Find similar images / search images by text" -> image embeddings (CLIP-style) + vector search
"Identify people by face" -> stop and read Section 5 first (legal, ethical, and access constraints)
"Process / resize / crop / draw on images" -> ImageSharp, SkiaSharp, or OpenCvSharp2. Image Classification
Assign a label (or probabilities over labels) to a whole image
Classification answers "what is this a picture of?" The output is a probability for each known class; you take the top one (or top-k) and apply a confidence threshold. This series' AI/ML Fundamentals guide covers the underlying concepts (training vs. inference, overfitting, precision/recall) that apply here unchanged.Option A: Run a pre-trained model with ONNX Runtime
Models trained on large general datasets (such as ImageNet's 1,000 classes) are available in ONNX format. Running one means loading the model, preprocessing the image exactly as the model expects, running inference, and decoding the output.// dotnet add package Microsoft.ML.OnnxRuntime
// dotnet add package SixLabors.ImageSharp
using Microsoft.ML.OnnxRuntime;
using Microsoft.ML.OnnxRuntime.Tensors;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.PixelFormats;
using SixLabors.ImageSharp.Processing;public sealed class ImageClassifier : IDisposable
{
private readonly InferenceSession _session; // create ONCE and reuse — loading a model is expensive
private readonly string _inputName;
private readonly string[] _labels;// ImageNet normalization used by many torchvision-trained models — MUST match YOUR model's training
private static readonly float[] Mean = [0.485f, 0.456f, 0.406f];
private static readonly float[] Std = [0.229f, 0.224f, 0.225f];public ImageClassifier(string modelPath, string labelsPath)
{
_session = new InferenceSession(modelPath);
_inputName = _session.InputMetadata.Keys.First();
_labels = File.ReadAllLines(labelsPath);
}public IReadOnlyList<(string Label, float Probability)> Classify(Image<Rgb24> image, int topK = 3)
{
// 1. PREPROCESS: resize + center-crop to 224x224, scale to 0..1, normalize, order as NCHW (batch, channel, height, width)
using var resized = image.Clone(x => x.Resize(new ResizeOptions { Size = new Size(224, 224), Mode = ResizeMode.Crop }));var tensor = new DenseTensor<float>([1, 3, 224, 224]);
resized.ProcessPixelRows(accessor =>
{
for (int y = 0; y < accessor.Height; y++)
{
Span<Rgb24> row = accessor.GetRowSpan(y);
for (int x = 0; x < row.Length; x++)
{
tensor[0, 0, y, x] = (row[x].R / 255f - Mean[0]) / Std[0];
tensor[0, 1, y, x] = (row[x].G / 255f - Mean[1]) / Std[1];
tensor[0, 2, y, x] = (row[x].B / 255f - Mean[2]) / Std[2];
}
}
});// 2. INFERENCE
using var results = _session.Run([NamedOnnxValue.CreateFromTensor(_inputName, tensor)]);
float[] logits = results.First().AsEnumerable<float>().ToArray();// 3. POSTPROCESS: softmax turns raw scores (logits) into probabilities that sum to 1
float max = logits.Max();
float[] exp = logits.Select(l => MathF.Exp(l - max)).ToArray(); // subtract max for numerical stability
float sum = exp.Sum();return exp.Select((e, i) => (Label: _labels[i], Probability: e / sum))
.OrderByDescending(x => x.Probability)
.Take(topK)
.ToList();
}public void Dispose() => _session.Dispose();
}THE PREPROCESSING MUST MATCH TRAINING. Check your model's documentation for:
- INPUT SIZE (224x224, 256x256, 384x384 ...)
- CHANNEL ORDER RGB vs BGR (OpenCV loads BGR; most deep-learning models expect RGB)
- LAYOUT NCHW (batch, channels, height, width) vs NHWC
- SCALING 0..1 vs 0..255 vs -1..1
- NORMALIZATION per-channel mean/std (or none)
- RESIZE POLICY stretch vs. preserve aspect ratio with crop or padding
A mismatch doesn't throw an error — it just silently degrades accuracy. This is the #1 cause of "the model works in Python
but gives garbage in .NET."Option B: Train your own classifier with ML.NET (transfer learning)
For your categories — product types, defect vs. OK, plant diseases — ML.NET can fine-tune the top of a pre-trained network on your labeled images. This is transfer learning: the pre-trained network already knows general visual features, so you need far less data (often hundreds of images per class rather than millions).// dotnet add package Microsoft.ML
// dotnet add package Microsoft.ML.Vision
// dotnet add package Microsoft.ML.ImageAnalytics
// dotnet add package SciSharp.TensorFlow.Redist (the TensorFlow runtime ML.NET's vision trainer uses)
using Microsoft.ML;
using Microsoft.ML.Vision;public class ImageData
{
public string ImagePath { get; set; } = "";
public string Label { get; set; } = ""; // e.g. the name of the folder the image sits in
}public class ImagePrediction
{
public string PredictedLabel { get; set; } = "";
public float[] Score { get; set; } = [];
}var mlContext = new MLContext(seed: 1);
// Folder layout: images/<label>/<file>.jpg — one subfolder per class
IEnumerable<ImageData> images = Directory.EnumerateFiles("images", "*.jpg", SearchOption.AllDirectories)
.Select(path => new ImageData { ImagePath = path, Label = Directory.GetParent(path)!.Name });IDataView data = mlContext.Data.ShuffleRows(mlContext.Data.LoadFromEnumerable(images));
// Convert the string label to a numeric "key" and load the raw image bytes
var preprocess = mlContext.Transforms.Conversion.MapValueToKey("LabelAsKey", nameof(ImageData.Label))
.Append(mlContext.Transforms.LoadRawImageBytes("Image", null, nameof(ImageData.ImagePath)));IDataView processed = preprocess.Fit(data).Transform(data);
var split = mlContext.Data.TrainTestSplit(processed, testFraction: 0.3, seed: 1);
var validation = mlContext.Data.TrainTestSplit(split.TestSet, testFraction: 0.5, seed: 1);
IDataView trainSet = split.TrainSet, validationSet = validation.TrainSet, testSet = validation.TestSet;var options = new ImageClassificationTrainer.Options
{
FeatureColumnName = "Image",
LabelColumnName = "LabelAsKey",
ValidationSet = validationSet,
Arch = ImageClassificationTrainer.Architecture.ResnetV2101, // the pre-trained network to build on
MetricsCallback = Console.WriteLine, // progress and validation accuracy per epoch
TestOnTrainSet = false,
ReuseTrainSetBottleneckCachedValues = true, // cache the expensive feature extraction between epochs
ReuseValidationSetBottleneckCachedValues = true,
};var pipeline = mlContext.MulticlassClassification.Trainers.ImageClassification(options)
.Append(mlContext.Transforms.Conversion.MapKeyToValue("PredictedLabel"));ITransformer model = pipeline.Fit(trainSet);
// Evaluate on images the model never trained on (see AI/ML Fundamentals, Section 8)
var predictions = model.Transform(testSet);
var metrics = mlContext.MulticlassClassification.Evaluate(predictions, labelColumnName: "LabelAsKey", predictedLabelColumnName: "PredictedLabel");
Console.WriteLine($"Micro-accuracy: {metrics.MicroAccuracy:P1} Macro-accuracy: {metrics.MacroAccuracy:P1}");mlContext.Model.Save(model, trainSet.Schema, "imageClassifier.zip");
(This follows the pattern of ML.NET's image-classification tutorials; the trainer's architecture options and required runtime packages change over time, so check the current ML.NET documentation. For deeper modeling or newer architectures, a common route is to train in PyTorch and export to ONNX, then run with ONNX Runtime as in Option A.)
Option C: Managed tagging
Azure AI Vision's Image Analysis returns general tags and captions for a photo with no training at all (Section 7) — a fine fit when "is there a dog in it?" level labeling suffices and your categories are common ones.Classification practicalities
CONFIDENCE THRESHOLDS A classifier always returns SOME label, even for an image unlike anything it has seen. Apply a threshold
and add an "unknown / needs human review" path — don't act on a 41% prediction as if it were certain.
CLASS IMBALANCE If 95% of images are "OK," accuracy is misleading (see AI/ML Fundamentals, Section 9). Track precision, recall,
and per-class results, and use a confusion matrix.
DATA QUALITY OVER MODEL Consistent labels and images representative of PRODUCTION conditions (lighting, angle, camera, backgrounds) matter
more than the architecture. Models trained on clean studio photos fail on blurry phone snapshots.
SPLIT BY SOURCE Near-duplicate images (frames of the same video, photos of the same object) must stay on one side of the
train/test split, or you'll measure memorization.
AUGMENTATION Flips, crops, brightness/contrast changes, and small rotations expand your data and improve robustness.
MULTI-LABEL If an image can have several labels at once, you need a multi-label setup (independent probabilities), not softmax.3. Object Detection
Find objects and their locations
Classification says what; object detection says what and where: a list of bounding boxes, each with a label and a confidence score. It powers counting, safety monitoring, inventory, cropping, and any workflow that needs to act on a specific region.Output of a detector for one image:
[ { label: "person", confidence: 0.91, box: (x=120, y=64, w=88, h=210) },
{ label: "car", confidence: 0.84, box: (x=410, y=190, w=260, h=130) }, ... ]Running a YOLO-style model with ONNX Runtime
YOLO-family models are the most common real-time detectors, and exported ONNX versions run well from .NET. The wiring has three parts that must all be right: letterbox preprocessing, inference, and postprocessing (decode boxes, filter by confidence, and apply non-maximum suppression).using Microsoft.ML.OnnxRuntime;
using Microsoft.ML.OnnxRuntime.Tensors;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.PixelFormats;
using SixLabors.ImageSharp.Processing;
using RectangleF = System.Drawing.RectangleF; // avoid ambiguity with ImageSharp's own RectangleFpublic record Detection(string Label, float Confidence, RectangleF Box);
public sealed class ObjectDetector : IDisposable
{
private const int InputSize = 640;
private readonly InferenceSession _session;
private readonly string _inputName;
private readonly string[] _labels;public ObjectDetector(string modelPath, string[] labels)
{
_session = new InferenceSession(modelPath);
_inputName = _session.InputMetadata.Keys.First();
_labels = labels;
}public IReadOnlyList<Detection> Detect(Image<Rgb24> original, float confThreshold = 0.25f, float iouThreshold = 0.45f)
{
// 1. LETTERBOX: scale to fit 640x640 while PRESERVING aspect ratio, padding the rest with gray
float scale = Math.Min((float)InputSize / original.Width, (float)InputSize / original.Height);
int newW = (int)(original.Width * scale), newH = (int)(original.Height * scale);
int padX = (InputSize - newW) / 2, padY = (InputSize - newH) / 2;using var resized = original.Clone(x => x.Resize(newW, newH));
var input = new DenseTensor<float>([1, 3, InputSize, InputSize]);
input.Fill(114f / 255f); // padding color
resized.ProcessPixelRows(accessor =>
{
for (int y = 0; y < accessor.Height; y++)
{
Span<Rgb24> row = accessor.GetRowSpan(y);
for (int x = 0; x < row.Length; x++)
{
input[0, 0, y + padY, x + padX] = row[x].R / 255f; // YOLO-style models typically want RGB scaled 0..1
input[0, 1, y + padY, x + padX] = row[x].G / 255f;
input[0, 2, y + padY, x + padX] = row[x].B / 255f;
}
}
});// 2. INFERENCE
using var results = _session.Run([NamedOnnxValue.CreateFromTensor(_inputName, input)]);
Tensor<float> output = results.First().AsTensor<float>(); // YOLOv8-style layout: [1, 4 + numClasses, numAnchors]int numClasses = output.Dimensions[1] - 4;
int numAnchors = output.Dimensions[2];// 3. POSTPROCESS: decode candidate boxes
var candidates = new List<Detection>();
for (int i = 0; i < numAnchors; i++)
{
int bestClass = 0; float bestScore = 0;
for (int c = 0; c < numClasses; c++)
{
float score = output[0, 4 + c, i];
if (score > bestScore) { bestScore = score; bestClass = c; }
}
if (bestScore < confThreshold) continue;float cx = output[0, 0, i], cy = output[0, 1, i], w = output[0, 2, i], h = output[0, 3, i];
// Undo the letterbox: model coordinates -> original image coordinates
float x1 = (cx - w / 2 - padX) / scale;
float y1 = (cy - h / 2 - padY) / scale;
candidates.Add(new Detection(_labels[bestClass], bestScore, new RectangleF(x1, y1, w / scale, h / scale)));
}// 4. NON-MAXIMUM SUPPRESSION: remove duplicate overlapping boxes for the same object
return NonMaxSuppression(candidates, iouThreshold);
}private static List<Detection> NonMaxSuppression(List<Detection> detections, float iouThreshold)
{
var kept = new List<Detection>();foreach (var group in detections.GroupBy(d => d.Label)) // per-class NMS
{
var remaining = group.OrderByDescending(d => d.Confidence).ToList();
while (remaining.Count > 0)
{
Detection best = remaining[0];
kept.Add(best);
remaining.RemoveAt(0);
remaining.RemoveAll(d => IoU(best.Box, d.Box) > iouThreshold);
}
}
return kept;
}private static float IoU(RectangleF a, RectangleF b)
{
RectangleF inter = RectangleF.Intersect(a, b);
float interArea = inter.IsEmpty ? 0 : inter.Width * inter.Height;
float union = a.Width * a.Height + b.Width * b.Height - interArea;
return union <= 0 ? 0 : interArea / union;
}public void Dispose() => _session.Dispose();
}The concepts behind that code
BOUNDING BOX A rectangle: (x, y, width, height) or (x1, y1, x2, y2). Models may output center-based (cx, cy, w, h) —
convert carefully.
CONFIDENCE The model's score for the detection; filter by a threshold. Lower = more detections and more false
positives; higher = fewer false positives but more misses.
IoU "Intersection over Union": the overlap between two boxes divided by their combined area (0 = no overlap,
1 = identical). Used both for NMS and for evaluating detections against ground truth.
NMS Detectors emit many overlapping boxes per object. NMS keeps the most confident box and discards others
that overlap it above the IoU threshold.
LETTERBOXING Preserving aspect ratio (padding) instead of stretching — stretched images distort object shapes and hurt
accuracy, and you MUST map boxes back to the original image's coordinates afterward.Model-specific details to verify
OUTPUT LAYOUT Differs by model family and export settings: some outputs include a separate "objectness" score, some are transposed
([1, numAnchors, 4 + classes]), and some exports bake NMS into the model. Inspect the output shape
(session.OutputMetadata) and the model's documentation — the code above matches the YOLOv8-style layout.
LABELS The class-name list must match the model's training (80 COCO classes for common pretrained checkpoints).
LICENSE Check the license of the model and its weights before commercial use — some popular detectors use licenses with
obligations (such as copyleft terms) that matter for products.
PERFORMANCE Indexing a Tensor per element is convenient but slow; for throughput, copy to a float[] and index with arithmetic.
Use GPU execution providers (CUDA, DirectML) and smaller or quantized (INT8) models where latency matters.Managed and trainable alternatives
AZURE AI VISION The Image Analysis API returns detected objects, people, and tags with bounding boxes — no model to host,
good for common categories (Section 7).
CUSTOM DETECTORS For YOUR objects (a specific part, a defect type), fine-tune a detector on labeled images (label with a tool like
CVAT or Label Studio), train in PyTorch or Azure ML, export to ONNX, and run with the code above.
ML.NET Offers object detection via Model Builder and Azure-based training workflows; check current documentation for
what it supports.Using detections
COUNTING Count boxes per class — but beware occlusion, tiny objects, and duplicates; validate counts against ground truth.
CROPPING Crop each box and send it to a second model (a classifier, OCR, or a face model) — a common multi-stage pipeline.
REGION RULES "Alert when a person is inside this polygon" — geometry on top of detections.
TRACKING (VIDEO) Detection works per frame. To follow an object across frames you need a tracking layer (e.g. SORT/ByteTrack-style
algorithms) that associates detections over time.
THRESHOLDS PER CLASS Tune confidence per class — a safety-critical class may deserve a lower threshold (accepting false alarms)
than a convenience feature.4. OCR
Turning pixels of text into text
Optical Character Recognition extracts text from images — scanned documents, photos of receipts and signs, screenshots, ID cards. Modern OCR returns not just the text but structure: lines and words with bounding boxes, reading order, and often tables and fields.The OCR options
AZURE AI VISION READ Cloud OCR for printed and handwritten text in photos and documents, many languages. Part of the
Image Analysis 4.0 API. Good for general "get me the text" needs.AZURE AI DOCUMENT Document-focused: layout (paragraphs, tables, selection marks), prebuilt models (invoices, receipts,
INTELLIGENCE IDs, business cards, tax forms) that return structured FIELDS, and custom models trained on your
forms. The right tool for documents rather than photos.TESSERACT Open-source engine, runs locally. Free and offline, but accuracy depends heavily on image quality and
preprocessing; weaker on photos, handwriting, and complex layouts.OTHER LOCAL ENGINES PaddleOCR (via .NET wrappers) and platform OCR (such as Windows' built-in OCR) are options with
different accuracy, language, and portability trade-offs.MULTIMODAL LLMs Vision-capable chat models can read text and extract structured fields from messy images
(Section 10) — flexible, but slower and costlier, and they can hallucinate text.Azure AI Vision: Read
// dotnet add package Azure.AI.Vision.ImageAnalysis
using Azure;
using Azure.AI.Vision.ImageAnalysis;var client = new ImageAnalysisClient(new Uri(endpoint), new AzureKeyCredential(key)); // or an Entra ID TokenCredential
ImageAnalysisResult result = await client.AnalyzeAsync(
BinaryData.FromBytes(imageBytes), // or pass a Uri to an image the service can fetch
VisualFeatures.Read);foreach (DetectedTextBlock block in result.Read.Blocks)
foreach (DetectedTextLine line in block.Lines)
{
Console.WriteLine(line.Text);foreach (DetectedTextWord word in line.Words)
Console.WriteLine($" {word.Text} (confidence {word.Confidence:P0})");
}Azure AI Document Intelligence: layout and prebuilt fields
// dotnet add package Azure.AI.DocumentIntelligence
using Azure;
using Azure.AI.DocumentIntelligence;var docClient = new DocumentIntelligenceClient(new Uri(endpoint), new AzureKeyCredential(key));
// "prebuilt-layout" returns text, tables, and structure; "prebuilt-invoice", "prebuilt-receipt", "prebuilt-idDocument"... return typed fields
Operation<AnalyzeResult> operation = await docClient.AnalyzeDocumentAsync(
WaitUntil.Completed, "prebuilt-invoice", BinaryData.FromBytes(pdfOrImageBytes));AnalyzeResult analysis = operation.Value;
foreach (AnalyzedDocument doc in analysis.Documents)
{
if (doc.Fields.TryGetValue("VendorName", out DocumentField? vendor))
Console.WriteLine($"Vendor: {vendor.ValueString} (confidence {vendor.Confidence:P0})");if (doc.Fields.TryGetValue("InvoiceTotal", out DocumentField? total))
Console.WriteLine($"Total: {total.ValueCurrency?.Amount}");
}(SDK method shapes for Document Intelligence have changed between releases; check the version you install.) Prebuilt models save enormous effort on common document types: instead of OCR-then-regex, you get named, typed fields with per-field confidence scores — the key to deciding which extractions need human review.
Tesseract: local OCR
// dotnet add package Tesseract (also download the trained-data files, e.g. eng.traineddata, into ./tessdata)
using Tesseract;using var engine = new TesseractEngine(@"./tessdata", "eng", EngineMode.Default);
using var pix = Pix.LoadFromFile("scan.png");
using var page = engine.Process(pix);Console.WriteLine(page.GetText());
Console.WriteLine($"Mean confidence: {page.GetMeanConfidence():P0}");OCR quality depends on the image — preprocess
RESOLUTION Aim for roughly 300 DPI equivalent for documents; upscale tiny text before OCR.
CONTRAST Convert to grayscale and apply adaptive thresholding (binarization) to separate text from uneven backgrounds.
DESKEW Rotate skewed scans so lines are horizontal. Correct orientation (EXIF rotation, upside-down pages).
DENOISE Remove speckle and scanner noise; avoid blurring text itself.
CROP Remove borders, shadows, and non-text regions.
LIGHTING Phone photos often have shadows and glare — normalize illumination.// Typical OpenCvSharp cleanup before OCR (Section 8)
using OpenCvSharp;using var src = Cv2.ImRead("receipt.jpg", ImreadModes.Color);
using var gray = new Mat();
Cv2.CvtColor(src, gray, ColorConversionCodes.BGR2GRAY);using var denoised = new Mat();
Cv2.GaussianBlur(gray, denoised, new Size(3, 3), 0);using var binary = new Mat();
Cv2.AdaptiveThreshold(denoised, binary, 255, AdaptiveThresholdTypes.GaussianC, ThresholdTypes.Binary, blockSize: 31, c: 15);Cv2.ImWrite("receipt-clean.png", binary);
Cloud services such as Read and Document Intelligence do much of this internally and are usually more forgiving than Tesseract; for local OCR, preprocessing is the difference between mediocre and good results.
Working with OCR output
CONFIDENCE Use word/line/field confidence to route low-confidence results to human review rather than trusting them blindly.
VALIDATE Cross-check extracted values: totals equal the sum of line items; dates parse; account numbers pass checksums;
VAT numbers match a pattern. Validation catches OCR errors that confidence scores miss.
STRUCTURE For tables and forms use a layout-aware service rather than reconstructing columns from raw text.
LANGUAGES / SCRIPTS Verify support for your languages and scripts, including right-to-left text and handwriting.
RAG PIPELINES OCR is the first stage of ingesting scanned PDFs for retrieval (see this series' RAG guide, Section 3): detect low-text PDF
pages and route them through OCR.
SENSITIVE DATA IDs, medical forms, and financial documents are sensitive. Control where images are sent, how long they're retained, and who
can see the extracted text (see the security notes in Section 11).Measuring OCR
Evaluate with character error rate (CER) and word error rate (WER) — the edit distance between OCR output and the ground-truth text, divided by the reference length — plus field-level accuracy for structured extraction:static double CharacterErrorRate(string reference, string hypothesis)
{
if (reference.Length == 0) return hypothesis.Length == 0 ? 0 : 1;
return (double)Levenshtein(reference, hypothesis) / reference.Length;
}static int Levenshtein(string a, string b)
{
var previous = Enumerable.Range(0, b.Length + 1).ToArray();
for (int i = 1; i <= a.Length; i++)
{
var current = new int[b.Length + 1];
current[0] = i;
for (int j = 1; j <= b.Length; j++)
current[j] = Math.Min(Math.Min(current[j - 1] + 1, previous[j] + 1), previous[j - 1] + (a[i - 1] == b[j - 1] ? 0 : 1));
previous = current;
}
return previous[b.Length];
}5. Face and Object Recognition
Be precise about what "face recognition" means
FACE DETECTION "Is there a face, and where?" — boxes (and optionally landmarks such as eyes and nose). The least sensitive
task; used for auto-cropping, blurring, camera focus, and counting.FACE VERIFICATION "Are these two images the same person?" (1:1). Used for unlocking a device or matching a selfie to an ID photo.
FACE IDENTIFICATION "Who is this, among a set of known people?" (1:N). Searches a gallery of enrolled faces. The most sensitive task.
FACE ATTRIBUTES Inferring age, gender, emotion, or other traits from a face. Scientifically contested and discontinued or
restricted by major providers (below). Avoid.Treat face recognition as a legal and ethical decision first
Faces are biometric data. Using them to identify people carries obligations and, in some jurisdictions and contexts, prohibitions. This is not a footnote:LAW AND REGULATION Biometric identification is treated as sensitive or special-category data under laws such as the GDPR, and some
jurisdictions have dedicated biometric statutes (for example Illinois' BIPA, with consent, retention, and
private-right-of-action provisions). The EU AI Act restricts or prohibits certain biometric uses — for example
real-time remote biometric identification in public spaces by law enforcement (with narrow exceptions), untargeted
scraping of facial images to build databases, and emotion recognition in workplaces and schools. Rules differ by
country and are evolving: consult legal counsel before building anything that identifies people by face.CONSENT AND PURPOSE Obtain informed consent where required, use faces only for the stated purpose, and don't repurpose them.
DATA PROTECTION Store face embeddings/templates as sensitive data: encryption, strict access control, minimal retention,
deletion on request, and a clear policy for what happens when a person withdraws consent.BIAS AND ACCURACY Face recognition accuracy has been shown to vary across demographic groups and imaging conditions. A system that
works well on average can fail badly — and unfairly — on specific groups. Evaluate by subgroup, set conservative
thresholds, and never make high-stakes decisions on a face match alone.PLATFORM POLICIES Microsoft's Azure Face service applies a Limited Access policy: identification and verification features require an
approved application and use case, and the earlier capabilities for inferring attributes like emotion, gender, and age
were retired. Other vendors have their own restrictions. Read the current terms.ALTERNATIVES Often a less invasive design meets the need: a badge or PIN plus a face DETECTION step for privacy blurring, 1:1
verification against a document the user provides, or on-device matching where templates never leave the user's device.The technical pipeline (for lawful, consented uses)
1. DETECT Find faces in the image (a face detector returns boxes and landmarks).
2. ALIGN Crop and rotate so eyes and mouth sit in standard positions — recognition models expect aligned faces.
3. EMBED A face-recognition network (ArcFace/FaceNet-style, available as ONNX models) converts each aligned face to a vector
(commonly 128-512 numbers) such that the same person's photos land close together.
4. COMPARE Verification: cosine similarity (or distance) between two embeddings vs. a threshold.
Identification: nearest-neighbor search over a gallery of enrolled embeddings (vector search — see this series'
Embeddings & Vector Search guide), then apply the threshold to the best match.
5. DECIDE A threshold trades false accepts against false rejects (below). Add liveness detection to defeat photos and replays.// Verification with embeddings from a face-recognition ONNX model (embedding extraction code is analogous to the classifier above)
float[] a = faceEmbedder.Embed(alignedFaceA);
float[] b = faceEmbedder.Embed(alignedFaceB);float similarity = TensorPrimitives.CosineSimilarity(a, b);
bool samePerson = similarity >= threshold; // choose the threshold from a labeled validation set, NOT from a blog postFALSE ACCEPT RATE (FAR) Different people judged to be the same — the SECURITY failure.
FALSE REJECT RATE (FRR) The same person judged different — the USABILITY failure.
The threshold trades one for the other. Pick it on a labeled validation set representative of your users and conditions, to meet a
target FAR for your risk level, and re-measure by subgroup.LIVENESS / ANTI-SPOOFING Without it, a printed photo or a screen replay can fool recognition. Use liveness checks for any
authentication use.Face detection in .NET (the low-risk task)
// Classic detector via OpenCvSharp — simple and dependency-light, but less accurate than modern deep-learning detectors
using OpenCvSharp;using var cascade = new CascadeClassifier("haarcascade_frontalface_default.xml");
using var image = Cv2.ImRead("group.jpg");
using var gray = new Mat();
Cv2.CvtColor(image, gray, ColorConversionCodes.BGR2GRAY);Rect[] faces = cascade.DetectMultiScale(gray, scaleFactor: 1.1, minNeighbors: 5, flags: HaarDetectionTypes.ScaleImage, minSize: new Size(40, 40));
foreach (Rect face in faces)
Cv2.Rectangle(image, face, Scalar.Red, 2); // or: blur the region to anonymize people in an imageModern deep-learning face detectors (such as the YuNet model supported by OpenCV's DNN module, or ONNX-exported detectors) are considerably more accurate than Haar cascades, especially for non-frontal and small faces; Azure AI Vision's people/face detection and the Azure Face service are managed options for detection.
"Object recognition": specific things, not just categories
CATEGORY RECOGNITION "That is a laptop" — classification/detection (Sections 2-3).
INSTANCE RECOGNITION "That is THIS exact product (SKU 4471)" or "this is a photo of the Eiffel Tower" — distinguishing specific instances.
The modern approach is EMBEDDING-BASED: embed a reference image of each known item, embed the query image, and
find the nearest reference via vector search (Section 6). Adding a new product means adding an embedding — no retraining.6. Image Embeddings
A vector that captures what an image means
An image embedding is a list of numbers (often hundreds to a thousand-plus) produced by a vision model such that visually or semantically similar images get nearby vectors — the same idea as text embeddings (see this series' Embeddings & Vector Search guide), applied to pixels. With embeddings you can compare, search, and cluster images without labeling anything.Multimodal embeddings: images and text in one space
Models in the CLIP family are trained on image–text pairs so that an image and its description land close together in the same vector space. That unlocks cross-modal search: embed a text query ("red running shoes on a trail") and find the nearest images.IMAGE → vector image embedding model
TEXT → vector text embedding model (the PAIRED model of the SAME family)
Compare with cosine similarity: text-to-image search, image-to-image search, zero-shot classification.Where embeddings come from
AZURE AI VISION MULTIMODAL EMBEDDINGS A managed API that embeds images AND text into a shared space (REST endpoints for vectorizing images and text).
No model to host.CLIP-STYLE MODELS VIA ONNX Open CLIP-family models exported to ONNX run locally with ONNX Runtime — an image encoder and a text
encoder (the text side needs the matching tokenizer, which is the fiddly part).VISION BACKBONES Features from image models such as ResNet or DINO-style networks work well for image-to-image similarity
(not text search) — export the model so its feature (pre-classification) layer is exposed.MULTIMODAL LLM PROVIDERS Some providers offer multimodal embedding endpoints; check availability.
A small abstraction
public interface IImageEmbedder
{
Task<float[]> EmbedImageAsync(byte[] imageBytes, CancellationToken ct = default);
Task<float[]> EmbedTextAsync(string text, CancellationToken ct = default); // only meaningful for multimodal (CLIP-style) models
}A managed implementation is a thin HTTP call (the exact URL, API version, and request shape for Azure AI Vision's multimodal embeddings should be taken from the current documentation):
public sealed class AzureVisionEmbedder(HttpClient http, string endpoint, string key) : IImageEmbedder
{
// Illustrative: confirm the route, api-version, and model-version in the current Azure AI Vision documentation
public async Task<float[]> EmbedImageAsync(byte[] imageBytes, CancellationToken ct = default)
{
using var request = new HttpRequestMessage(HttpMethod.Post,
$"{endpoint}/computervision/retrieval:vectorizeImage?api-version=2024-02-01&model-version=2023-04-15")
{
Content = new ByteArrayContent(imageBytes) { Headers = { ContentType = new("application/octet-stream") } }
};
request.Headers.Add("Ocp-Apim-Subscription-Key", key);using var response = await http.SendAsync(request, ct);
response.EnsureSuccessStatusCode();var json = await response.Content.ReadFromJsonAsync<EmbeddingResponse>(cancellationToken: ct);
return json!.Vector;
}public async Task<float[]> EmbedTextAsync(string text, CancellationToken ct = default) { /* the vectorizeText counterpart */ throw new NotImplementedException(); }
private record EmbeddingResponse(float[] Vector);
}What you can build
// 1. INDEX: embed every catalog image once and store the vector (with the image id and metadata) in a vector store
foreach (var product in catalog)
await store.UpsertAsync(product.Id, await embedder.EmbedImageAsync(product.ImageBytes), product.Metadata);// 2. SEARCH BY IMAGE: "find products that look like this photo"
float[] queryVector = await embedder.EmbedImageAsync(uploadedPhotoBytes);
var similar = await store.SearchAsync(queryVector, top: 10, filter: tenantFilter);// 3. SEARCH BY TEXT (multimodal models only): "red leather handbag"
float[] textVector = await embedder.EmbedTextAsync("red leather handbag");
var matches = await store.SearchAsync(textVector, top: 10, filter: tenantFilter);// 4. ZERO-SHOT CLASSIFICATION: compare the image to text prompts for each candidate label — no training data
string[] labels = ["cat", "dog", "bird"];
var labelVectors = await Task.WhenAll(labels.Select(l => embedder.EmbedTextAsync($"a photo of a {l}")));
float[] imageVector = await embedder.EmbedImageAsync(imageBytes);
int best = Enumerable.Range(0, labels.Length).MaxBy(i => TensorPrimitives.CosineSimilarity(imageVector, labelVectors[i]));USE CASES
Visual search ("shop the look"), reverse image search Duplicate and near-duplicate detection (moderation, DAM systems)
Recommendations ("more like this") Instance recognition (match a photo to a known product)
Clustering/organizing photo collections Zero-shot classification and tagging
Retrieval for multimodal RAG (find relevant diagrams or figures)Rules (the same as for text embeddings)
SAME MODEL for the index and the queries — different models' vectors are not comparable. Record the model name/version with stored vectors; changing
models means re-embedding everything.
NORMALIZE / USE COSINE per the model's guidance; choose the vector index metric to match.
CACHE embeddings by content hash — re-embedding unchanged images is pure waste.
FILTER BY METADATA and permissions INSIDE the vector query (tenant, category, date) — see the vector search guide.
EVALUATE retrieval with real queries and labeled relevance (Recall@k), not by eyeballing a few results.
LIMITS: embeddings capture overall semantic/visual similarity, not fine detail. They can miss small text in an image, exact counts, or subtle defects —
pair them with OCR or detection when those matter.
PRIVACY: embeddings derived from sensitive images (faces, medical, documents) are sensitive data.7. Azure AI Vision
Microsoft's managed vision services
AZURE AI VISION — IMAGE ANALYSIS 4.0 General image understanding: CAPTIONS (a sentence describing the image), DENSE CAPTIONS (regions),
TAGS, OBJECT DETECTION, PEOPLE DETECTION, SMART CROPS, and READ (OCR). One API, choose features per call.AZURE AI VISION — MULTIMODAL Image and text embeddings for vector search (Section 6).
EMBEDDINGSAZURE AI DOCUMENT INTELLIGENCE Document OCR, layout, prebuilt extractors (invoice, receipt, ID, ...) and custom models (Section 4).
AZURE AI FACE Face detection and, for approved use cases, verification/identification (Section 5; Limited Access applies).
AZURE OPENAI (VISION-CAPABLE MODELS) Multimodal chat models that understand images (Section 10).
AZURE AI CONTENT SAFETY Detects harmful content in images and text — useful for moderating user uploads.
CONTAINERS Some Vision and Document Intelligence capabilities are available as containers you can run in your own
environment for data-residency or disconnected scenarios (check which features and licensing apply).Availability, regions, and feature maturity vary, and Microsoft has retired or announced retirement of older offerings (earlier Computer Vision API versions, Custom Vision, and some restricted Face features among them). Check the current service documentation and retirement notices before depending on any feature.
Image Analysis 4.0 in C
// dotnet add package Azure.AI.Vision.ImageAnalysis
using Azure;
using Azure.AI.Vision.ImageAnalysis;var client = new ImageAnalysisClient(new Uri(endpoint), new AzureKeyCredential(key));
ImageAnalysisResult result = await client.AnalyzeAsync(
BinaryData.FromBytes(imageBytes),
VisualFeatures.Caption | VisualFeatures.Tags | VisualFeatures.Objects | VisualFeatures.People | VisualFeatures.Read,
new ImageAnalysisOptions { GenderNeutralCaption = true, Language = "en" });Console.WriteLine($"Caption: {result.Caption.Text} ({result.Caption.Confidence:P0})");
foreach (DetectedTag tag in result.Tags.Values)
Console.WriteLine($"Tag: {tag.Name} ({tag.Confidence:P0})");foreach (DetectedObject obj in result.Objects.Values)
Console.WriteLine($"Object: {obj.Tags[0].Name} ({obj.Tags[0].Confidence:P0}) at X={obj.BoundingBox.X}, Y={obj.BoundingBox.Y}, W={obj.BoundingBox.Width}, H={obj.BoundingBox.Height}");foreach (DetectedPerson person in result.People.Values.Where(p => p.Confidence > 0.6))
Console.WriteLine($"Person at {person.BoundingBox.X},{person.BoundingBox.Y} ({person.Confidence:P0})");You can pass a
Uriinstead of bytes when the service can fetch the image itself. Request only the visual features you need — each feature adds to the response and the bill, and some (such as captions) are available only in certain regions.Authentication, networking, and operations
AUTH Prefer Microsoft Entra ID (DefaultAzureCredential, managed identity) over API keys; if you use keys, keep them in Key Vault.
(The SDK clients accept either.)
NETWORKING Private endpoints and virtual-network restrictions are available for enterprise deployments.
LIMITS Image size and dimension limits, supported formats, and requests-per-second quotas apply — resize before upload, and handle
HTTP 429 with backoff (see AI APIs with .NET, Sections 10-11). The SDK clients include retry policies for transient errors.
COST Billed per transaction (per feature, per image). Cache results by image hash, don't re-analyze unchanged images, and request only
needed features.
DATA Understand what the service retains and where it processes data; images can be sensitive. Use containers or private networking if policy requires.Wire it through your own abstraction
public interface IImageAnalyzer
{
Task<ImageSummary> AnalyzeAsync(byte[] image, CancellationToken ct);
}public record ImageSummary(string Caption, IReadOnlyList<string> Tags, IReadOnlyList<(string Label, double Confidence, int X, int Y, int W, int H)> Objects, string Text);
Hide the Azure types behind your own interface so you can swap in a self-hosted model or a different provider, add caching and resilience in one place, and unit test with a fake — the same decoupling argument made for AI services in Microsoft.Extensions.AI.
8. OpenCV Integration
OpenCV: the classic computer vision toolkit
OpenCV is a mature, high-performance library for image and video processing and "classical" computer vision: filtering, geometric transforms, edge and contour detection, feature matching, camera calibration, video I/O, background subtraction, tracking, and a DNN module that can run neural networks. In .NET you use it through OpenCvSharp (the most popular open-source wrapper) or Emgu CV (another wrapper with its own licensing terms to review).Setup
dotnet add package OpenCvSharp4
dotnet add package OpenCvSharp4.runtime.win # native OpenCV binaries — choose the runtime package for YOUR platform:
# .runtime.win, .runtime.ubuntu.20.04-x64, .runtime.linux-arm, .runtime.osx ...
dotnet add package OpenCvSharp4.Extensions # conversions to/from System.Drawing bitmaps (Windows-oriented)The managed package is a wrapper; native binaries must be present at runtime, which is the main deployment gotcha (especially in Docker/Linux — install the matching runtime package and any required system libraries, and test in the actual container image).
The
Matand the dispose rule
using OpenCvSharp;// Mat is OpenCV's image/matrix type. It owns NATIVE memory that the .NET garbage collector cannot see —
// ALWAYS dispose it (using), especially in loops and servers, or you will leak memory.
using Mat image = Cv2.ImRead("photo.jpg", ImreadModes.Color); // loads as BGR (not RGB!)
using Mat gray = new();
Cv2.CvtColor(image, gray, ColorConversionCodes.BGR2GRAY);using Mat blurred = new();
Cv2.GaussianBlur(gray, blurred, new Size(5, 5), 0);using Mat edges = new();
Cv2.Canny(blurred, edges, threshold1: 50, threshold2: 150);Cv2.ImWrite("edges.png", edges);
Colors are BGR — the classic bug
OpenCV stores color images as BLUE-GREEN-RED, while most deep-learning models (and most other imaging libraries, including ImageSharp and
SkiaSharp) use RGB. Feed an unconverted BGR image to an RGB-trained model and accuracy silently drops. Convert with
Cv2.CvtColor(src, dst, ColorConversionCodes.BGR2RGB) at the boundary.Common operations
Cv2.Resize(image, resized, new Size(640, 480)); // resize
Cv2.Rotate(image, rotated, RotateFlags.Rotate90Clockwise); // rotate
using var roi = new Mat(image, new Rect(100, 50, 200, 200)); // crop (a view into the same data)// Find contours (shapes) in a binary image
Cv2.FindContours(binary, out Point[][] contours, out HierarchyIndex[] hierarchy, RetrievalModes.External, ContourApproximationModes.ApproxSimple);
foreach (var contour in contours.Where(c => Cv2.ContourArea(c) > 500))
Cv2.Rectangle(image, Cv2.BoundingRect(contour), Scalar.Lime, 2);// Draw and label detections from Section 3
foreach (var d in detections)
{
var rect = new Rect((int)d.Box.X, (int)d.Box.Y, (int)d.Box.Width, (int)d.Box.Height);
Cv2.Rectangle(image, rect, Scalar.Red, 2);
Cv2.PutText(image, $"{d.Label} {d.Confidence:P0}", new Point(rect.X, Math.Max(15, rect.Y - 5)),
HersheyFonts.HersheySimplex, 0.6, Scalar.Red, 2);
}Moving between OpenCV, bytes, and other libraries
// Encode/decode in memory (no temp files) — the way to move images over HTTP or through queues
Cv2.ImEncode(".jpg", image, out byte[] jpegBytes);
using Mat decoded = Cv2.ImDecode(jpegBytes, ImreadModes.Color);// To feed a neural-network input tensor from a Mat, copy to a managed array and convert BGR -> RGB deliberately (not by accident)
Video: frame loops
using var capture = new VideoCapture("traffic.mp4"); // a file path, an RTSP/HTTP URL, or a camera index (0)
using var frame = new Mat();int frameIndex = 0;
while (capture.Read(frame) && !frame.Empty())
{
// Detection is expensive: run it every Nth frame (and track in between) instead of every frame
if (frameIndex++ % 5 == 0)
{
// convert and run a detector (Section 3) ...
}
}VIDEO PRACTICALITIES
- Decode on a worker thread; keep a small bounded queue between capture, inference, and output so a slow model drops frames instead of
accumulating latency and memory.
- Process every Nth frame or at reduced resolution for real-time budgets.
- Use hardware decoding and GPU inference when throughput matters.
- Reconnect automatically for network streams; they drop.
- Watch resource use: each frame is a large native allocation.OpenCV's DNN module
using OpenCvSharp.Dnn;using Net net = CvDnn.ReadNetFromOnnx("model.onnx");
using Mat blob = CvDnn.BlobFromImage(image, scaleFactor: 1 / 255.0, size: new Size(640, 640), mean: new Scalar(), swapRB: true, crop: false);
net.SetInput(blob);
using Mat output = net.Forward();OpenCV can run many ONNX models directly (convenient for simple deployments, and for models such as YuNet face detection). For broader operator support, GPU execution providers, and performance tuning, ONNX Runtime is usually the stronger inference engine; many teams use OpenCV for image I/O and preprocessing, ONNX Runtime for inference.
When to reach for OpenCV
VIDEO AND CAMERAS capture, decode, encode, streams
GEOMETRIC / CLASSICAL CV perspective correction, deskewing, contour and shape analysis, feature matching, calibration, background subtraction
PREPROCESSING FOR OCR grayscale, thresholding, denoising, deskew (Section 4)
FAST NATIVE IMAGE OPS heavy per-pixel work where native performance matters
Prefer a managed library (Section 9) for simple resizing, format conversion, and drawing in web back ends — fewer native dependencies to deploy.9. Image Processing APIs
Choosing a library for decoding, transforming, drawing, and encoding
Before and after any model, you decode images, orient and resize them, convert formats, and sometimes draw annotations. .NET offers several libraries, with important differences in platform support and licensing.SixLabors.ImageSharp Fully managed, cross-platform, modern API, no native dependencies, good for web servers and containers.
Drawing lives in the separate ImageSharp.Drawing package. NOTE: it uses a "split license" — free for open-source and
qualifying small organizations, but commercial use may require a paid license; check the current terms.SkiaSharp Bindings to Google's Skia graphics engine; fast, widely used (MAUI, Avalonia), strong 2D drawing and text rendering. Native
binaries per platform; permissive MIT license.System.Drawing.Common GDI+ based. On .NET 6+ it is supported on WINDOWS ONLY. Don't use it for cross-platform or Linux-container services.
Magick.NET Wrapper over ImageMagick: enormous format support and many operations; native dependencies; Apache-style license.
NetVips Wrapper over libvips: very fast and memory-efficient for large images and pipelines; native dependency.
OpenCvSharp Computer vision and video, with image I/O and drawing (Section 8). BGR channel order.
Windows imaging APIs WIC/Windows.Graphics.Imaging for Windows-only apps.
ImageSharp: the everyday operations
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.Formats.Jpeg;
using SixLabors.ImageSharp.Processing;// Load -> fix orientation -> resize (never enlarge) -> strip metadata -> save
using Image image = await Image.LoadAsync(inputStream);image.Mutate(x => x
.AutoOrient() // apply the EXIF rotation phones store instead of rotating pixels
.Resize(new ResizeOptions { Size = new Size(1600, 1600), Mode = ResizeMode.Max })); // fit within 1600x1600, keep aspect ratioimage.Metadata.ExifProfile = null; // remove EXIF (GPS location, device info) before storing or sharing
await image.SaveAsJpegAsync(outputStream, new JpegEncoder { Quality = 85 });
Cross-cutting image gotchas
EXIF ORIENTATION Phone photos are often stored sideways with an orientation flag. If you ignore it, the model sees a rotated image (and OCR fails).
Apply orientation (AutoOrient) BEFORE inference.EXIF PRIVACY Photos carry GPS coordinates, timestamps, and device details. Strip metadata before storing or publishing images you didn't create.
COLOR ORDER RGB (ImageSharp, SkiaSharp, most models) vs BGR (OpenCV). Convert deliberately.
COLOR SPACES Some images are CMYK, grayscale, 16-bit, or have alpha. Convert to a known format (e.g. Rgb24) before building tensors.
RESIZE QUALITY Choose a good resampler (bicubic/Lanczos) for downscaling; the choice can measurably affect model accuracy, and it should match
training if the model is sensitive.MEMORY A 50-megapixel image decodes to ~150 MB of pixels. Cap dimensions BEFORE full decode (read just the header with Image.Identify) and
resize early.THREAD SAFETY Image/Mat objects are not thread-safe; don't share one across concurrent requests.
FORMAT SUPPORT JPEG, PNG, WebP, GIF, BMP, TIFF are common; HEIC (iPhone) and RAW formats often need extra libraries — plan for them if users upload from phones.
Draw on images: ImageSharp.Drawing and SkiaSharp
// SkiaSharp: draw labeled boxes on a decoded bitmap
using SkiaSharp;using SKBitmap bitmap = SKBitmap.Decode(imageBytes);
using var canvas = new SKCanvas(bitmap);
using var boxPaint = new SKPaint { Color = SKColors.Red, Style = SKPaintStyle.Stroke, StrokeWidth = 3 };foreach (var d in detections)
canvas.DrawRect(d.Box.X, d.Box.Y, d.Box.Width, d.Box.Height, boxPaint);using var image = SKImage.FromBitmap(bitmap);
using var data = image.Encode(SKEncodedImageFormat.Jpeg, 85);
await File.WriteAllBytesAsync("annotated.jpg", data.ToArray());Validate uploaded images — they are untrusted input
// Read dimensions from the header WITHOUT decoding the whole image, and reject abusive inputs early
ImageInfo info = await Image.IdentifyAsync(stream); // throws on unrecognized formats
if (info.Width > 10_000 || info.Height > 10_000 || (long)info.Width * info.Height > 50_000_000)
return Results.BadRequest("Image dimensions are too large.");
stream.Position = 0;Treat image uploads like any untrusted file: enforce a maximum byte size (and request-body limits); verify the actual format from content, not the file extension
or Content-Type header; limit decoded dimensions (decompression bombs — a tiny file that expands to gigabytes of pixels); process in a resource-limited
context; re-encode to a safe format rather than storing originals blindly; and scan for malware where policy requires. Keep image-decoding libraries updated —
image codecs are a historically common source of vulnerabilities.10. Multimodal LLMs vs. Classical Computer Vision
Vision-capable chat models are now a mainstream option
Modern chat models accept images alongside text. ThroughMicrosoft.Extensions.AI(see this series' Microsoft.Extensions.AI guide), sending an image is just another content item in a message:using Microsoft.Extensions.AI;
var message = new ChatMessage(ChatRole.User,
[
new TextContent("Extract the vendor name, invoice number, date, and total from this invoice. Respond as JSON."),
new DataContent(imageBytes, "image/jpeg"),
]);ChatResponse response = await chatClient.GetResponseAsync([message]);
// Or get a typed result directly
public record InvoiceFields(string Vendor, string InvoiceNumber, string Date, decimal Total);
ChatResponse<InvoiceFields> typed = await chatClient.GetResponseAsync<InvoiceFields>([message]);
InvoiceFields invoice = typed.Result; // validate it — shape is guaranteed, truth is notWhat multimodal LLMs are good at
OPEN-ENDED QUESTIONS "Is anything unsafe in this photo?" "Describe what happened in this screenshot." — no model training or fixed label set.
ZERO-SHOT FLEXIBILITY New categories or extraction schemas by changing the prompt, not retraining.
MESSY DOCUMENTS Irregular layouts, handwriting, receipts photographed at angles, charts and diagrams.
REASONING OVER IMAGES Combining visual content with text instructions and context.
RAPID PROTOTYPING Proof of concept in an afternoon.Where they fall short
HALLUCINATION They can confidently invent text, numbers, or objects that aren't in the image. For OCR-like extraction this is dangerous: a plausible but
wrong total looks exactly like a right one.
PRECISE LOCALIZATION Bounding boxes, pixel-accurate positions, and exact counting of many objects are unreliable compared with dedicated detectors.
CONSISTENCY Non-deterministic; the same image can yield different outputs.
COST AND LATENCY Each image consumes many tokens and takes seconds — painful at video frame rates or millions of images.
THROUGHPUT Rate limits and per-call pricing don't suit high-volume, real-time pipelines.
PRIVACY Images are sent to a hosted model; sensitive content (faces, IDs, medical images) raises data-protection questions.
LIMITS Image size and count per request, and resolution that may downscale small text.Choosing
NeedPreferPrecise boxes, counts, real-time videoDedicated detector (ONNX/YOLO)High-volume, low-cost, consistent labeling of fixed categoriesTrained classifier or detectorStructured fields from standard documents with confidence scoresDocument Intelligence prebuilt/custom modelsPlain text from many images, cheaplyOCR (Vision Read / Tesseract)Flexible, varied, open-ended image questions or extraction schemasMultimodal LLMImage similarity / search at scaleEmbeddings + vector searchRapid prototype to learn what's possibleMultimodal LLM, then replace pieces with specialized models if cost/accuracy demandHybrid pipelines are often best
Classical + LLM, each doing what it's best at:
- OCR / Document Intelligence extracts the text and fields WITH confidence; an LLM interprets or normalizes them — or only handles the low-confidence cases.
- A detector finds and crops regions; an LLM describes or reads each crop.
- A cheap classifier screens images; only the uncertain ones go to the expensive multimodal model.
- An LLM proposes; deterministic code VALIDATES (totals add up, dates parse, values within ranges).All the cautions from AI Agents and LLM Application Development apply: treat text inside images as untrusted (images can contain visual prompt injection — instructions written into a picture), validate outputs, and keep a human in the loop for consequential decisions.
11. Building a Vision Service in ASP.NET Core
A detection endpoint, with the production concerns included
// Program.cs
builder.Services.AddSingleton(sp => new ObjectDetector("models/yolov8n.onnx", CocoLabels.All)); // load the model ONCEbuilder.Services.AddRateLimiter(o => o.AddPolicy("vision", ctx => RateLimitPartition.GetFixedWindowLimiter(
ctx.User.Identity?.Name ?? "anonymous", _ => new FixedWindowRateLimiterOptions { PermitLimit = 30, Window = TimeSpan.FromMinutes(1) })));builder.WebHost.ConfigureKestrel(k => k.Limits.MaxRequestBodySize = 10_000_000); // cap upload size
var inference = new SemaphoreSlim(Environment.ProcessorCount); // bound concurrent CPU-heavy inference
app.MapPost("/detect", async (IFormFile file, ObjectDetector detector, CancellationToken ct) =>
{
if (file.Length is 0 or > 10_000_000) return Results.BadRequest("Image must be between 1 byte and 10 MB.");await using Stream stream = file.OpenReadStream();
ImageInfo info;
try { info = await Image.IdentifyAsync(stream, ct); } // verify it's a real image; read dimensions cheaply
catch (Exception) { return Results.BadRequest("Unsupported or corrupt image."); }if ((long)info.Width * info.Height > 50_000_000) return Results.BadRequest("Image dimensions are too large.");
stream.Position = 0;using Image<Rgb24> image = await Image.LoadAsync<Rgb24>(stream, ct);
image.Mutate(x => x.AutoOrient()); // honor EXIF rotation BEFORE inferenceawait inference.WaitAsync(ct);
try
{
var detections = await Task.Run(() => detector.Detect(image), ct); // keep the request thread free; bound the work
return Results.Ok(detections.Select(d => new { d.Label, d.Confidence, d.Box.X, d.Box.Y, d.Box.Width, d.Box.Height }));
}
finally { inference.Release(); }
})
.RequireAuthorization()
.RequireRateLimiting("vision")
.DisableAntiforgery(); // an API endpoint, not a browser form postArchitecture and performance considerations
LOAD MODELS ONCE Register the inference session as a singleton; loading a model per request is slow and memory-hungry. InferenceSession.Run
is safe to call concurrently; image objects are not shared.WARM UP The first inference is slower (graph optimization, allocations). Run a dummy inference at startup so users don't pay for it.
BOUND CONCURRENCY CPU/GPU inference is resource-intensive; limit parallel work (SemaphoreSlim) and queue the rest rather than thrashing.
ASYNC AND BACKGROUND For slow or bulk jobs (video, large batches, heavy OCR), accept the upload, enqueue a job, return an ID, and let a background worker process it
(BackgroundService / queue / Azure Functions), with status polling or a webhook/SignalR notification.GPU Microsoft.ML.OnnxRuntime.Gpu (CUDA) or .DirectML (Windows) can accelerate inference substantially; build and test GPU container images
deliberately (drivers, CUDA/cuDNN versions).MODEL SIZE VS. SPEED Smaller or quantized models (INT8) and lower input resolutions cut latency and memory at some accuracy cost; measure the trade-off.
BATCHING Batching several images into one inference call raises GPU throughput at the cost of per-request latency.
CACHE Key results by a hash of the image bytes (plus model version) — repeated images are common (duplicates, retries, shared files).
MODEL MANAGEMENT Version models like code; store them in blob storage or the image, record the model version with each result, and use a rolling
rollout so you can compare and roll back.OBSERVABILITY Log latency, input sizes, confidence distributions, and error rates; export traces/metrics (OpenTelemetry). Drift in confidence
distributions is an early warning that production images have changed.DOCKER Include native dependencies (OpenCV runtime packages, ONNX Runtime native libs) and test the actual image; keep images small.
Security and privacy for image services
UNTRUSTED UPLOADS Size/dimension limits, format verification, resource-limited processing, updated codecs (Section 9).
AUTHORIZATION Users may only analyze and retrieve THEIR images/results; scope storage and queries per user/tenant.
SENSITIVE CONTENT Images may contain faces, IDs, medical or financial documents. Minimize retention, encrypt at rest and in transit, restrict who and what
can see them, and be explicit about when images leave your environment (cloud APIs, LLMs).
EXIF AND METADATA Strip location and device data before storing or sharing.
MODERATION If users upload content others will see, screen it (for example with Azure AI Content Safety).
ADVERSARIAL INPUTS Models can be fooled by crafted images (adversarial examples); don't rely on vision output alone for security-critical decisions. Images can
also carry hidden instructions aimed at multimodal LLMs.
COSTS AND ABUSE Rate limit, cap sizes, and authenticate — an open image-analysis endpoint (especially one calling a paid API) is a cost-abuse target.12. Evaluating Vision Systems
Pick metrics that match the task — then test on realistic data
CLASSIFICATION Accuracy (careful with imbalance), precision, recall, F1, per-class results, confusion matrix, calibration of confidence scores.
(See AI/ML Fundamentals, Section 9.)OBJECT DETECTION IoU between predicted and true boxes; a detection is correct if the label matches and IoU exceeds a threshold (commonly 0.5).
PRECISION = correct detections / all detections; RECALL = correct detections / all true objects.
AP (average precision) per class summarizes the precision-recall curve; mAP averages across classes (and sometimes IoU thresholds).
Also check counting error and per-size performance (small objects are hardest).OCR Character error rate (CER) and word error rate (WER) for text; FIELD-LEVEL accuracy and per-field confidence calibration for extraction;
end-to-end "is the final record correct?" rate.FACE VERIFICATION FAR and FRR at the chosen threshold, ROC curves, and results BROKEN DOWN BY DEMOGRAPHIC GROUP and imaging condition.
EMBEDDINGS / Recall@k and MRR on labeled query-to-relevant-image pairs (see Embeddings & Vector Search and RAG, Section 12).
VISUAL SEARCHLLM VISION Task-specific accuracy against ground truth, plus hallucination checks (does the extracted value actually appear in the image?),
consistency across repeated runs, and cost/latency.Data practices that make evaluation honest
REPRESENTATIVE TEST SET Collect images from the REAL production environment: your cameras, lighting, devices, angles, resolutions, and edge cases (blur, glare,
occlusion, odd orientations). Models validated on clean benchmark photos routinely disappoint in the field.
SPLIT BY SOURCE Keep near-duplicates (frames from one video, shots of the same object, pages of one document) on one side of the split.
CONSISTENT LABELS Ambiguous or noisy labels cap achievable accuracy. Write labeling guidelines, measure inter-annotator agreement, and review disputed cases.
STRATIFY AND SLICE Report results by condition (day/night, indoor/outdoor, device type), by class, and by object size — averages hide failures.
FAIRNESS Where people appear, evaluate across demographic groups; large gaps are a defect even when the average looks good.
NEGATIVE EXAMPLES Include images with NONE of the target objects, and out-of-distribution images, to measure false positives and unknown handling.
FROZEN TEST SET Don't tune on it. Keep a held-out set untouched until the end, and refresh it as the world changes.In production
CONFIDENCE DISTRIBUTIONS Track them over time; shifts signal that production images no longer match what the model was validated on.
HUMAN REVIEW QUEUE Route low-confidence or high-stakes results to people; use their corrections as new labeled data.
SAMPLED AUDIT Regularly review random samples of automated decisions, not just the flagged ones.
FEEDBACK LOOP Capture errors and overrides, label them, and fold them into the next training and evaluation set.
VERSIONED REGRESSION SUITE Re-run your evaluation set whenever you change models, thresholds, preprocessing, or a provider's model version.
COST AND LATENCY BUDGETS Track p50/p95 latency and per-image cost alongside accuracy.13. Choosing the Right Approach
START WITH THE REQUIREMENTS, NOT THE MODEL
What decision does this feed? What does a mistake cost? Is it real-time? How many images per day? Do images contain sensitive data? Must it run offline or
inside a boundary? Are the categories common or specific to you?THEN WALK THE LADDER, STOPPING AS EARLY AS POSSIBLE
1. A managed API handles it (Azure AI Vision, Document Intelligence) -> build in days; pay per call
2. A multimodal LLM handles it well enough -> flexible and fast to build; check cost, latency, and hallucination risk
3. A pre-trained ONNX model handles it -> run locally; no per-call fee; you own pre/postprocessing
4. You need custom accuracy on your own data -> labeled dataset, transfer learning or fine-tuning, evaluation, MLOpsTHEN HARDEN
ScenarioGood starting pointExtract fields from invoices/receipts/IDsDocument Intelligence prebuilt models + validation + review of low-confidence fieldsCaption/tag user photos for accessibility or searchAzure AI Vision Image Analysis or a multimodal LLMCount people or vehicles in camera feedsYOLO-style ONNX detector (+ tracking), GPU if real-timeClassify manufacturing defectsLabeled data + transfer learning (ML.NET or PyTorch → ONNX), strict evaluation"Find products that look like this photo"Image embeddings + vector searchRead text from phone photos of documentsVision Read or Document Intelligence; preprocess; validateBlur faces for privacyFace DETECTION (not recognition) + blurringAuthenticate users by faceLegal review first; consented 1:1 verification with liveness, or avoid biometricsUnderstand messy, varied images with flexible questionsMultimodal LLM with validation
Evaluate on realistic images, set thresholds and a human-review path, add validation, secure the pipeline, monitor drift, and plan for model and service lifecycle changes.14. Common Pitfalls
PitfallWhy it hurtsBetter approachPreprocessing that doesn't match training (size, channel order, scaling, normalization)Accuracy silently collapses with no errorCheck the model's docs; test against reference outputs from the original framework (Section 2)Mixing BGR (OpenCV) and RGB (models, ImageSharp)Colors swapped; degraded predictionsConvert deliberately at the boundary (Sections 2, 8, 9)Ignoring EXIF orientationSideways images give bad detections and OCRAutoOrient()before inference (Sections 9, 11)Forgetting to disposeMatand imagesNative memory leaks that crash serversusingeverywhere; dispose in loops (Section 8)Loading the model per requestSlow, memory-hungrySingleton inference session; warm up at startup (Section 11)Sharing anImage/Matacross threadsCorruption and racesOne per request; share only the thread-safe inference session (Section 9)Using System.Drawing.Common on LinuxWindows-only on modern .NETImageSharp, SkiaSharp, or OpenCvSharp (Section 9)Skipping NMS or the IoU/threshold tuningDuplicate boxes; too many false alarms or missesApply per-class NMS; tune thresholds on a validation set (Section 3)Not mapping boxes back through letterboxingBoxes land in the wrong place on the original imageUndo padding and scale in postprocessing (Section 3)Trusting every predictionA classifier always returns a label, even for unfamiliar imagesConfidence thresholds, an "unknown" path, human review (Section 2)Reporting accuracy on imbalanced dataA trivial majority-class model scores highPrecision, recall, F1, per-class metrics, confusion matrix (Sections 2, 12)Test images unlike productionGreat benchmarks, poor field performanceEvaluate on representative, sliced real-world images (Section 12)Train/test leakage from near-duplicate imagesInflated scores that measure memorizationSplit by source (Sections 2, 12)Feeding raw scans to OCR without cleanupLow accuracy, especially with TesseractResolution, contrast, deskew, denoise; prefer cloud/layout OCR for hard inputs (Section 4)Trusting OCR/LLM-extracted values without validationPlausible but wrong numbers flow into systemsConfidence routing plus business-rule validation (Sections 4, 10)Using a multimodal LLM for precise counting or boxesUnreliable localization; hallucinationUse a dedicated detector; use the LLM for interpretation (Section 10)Building face identification without a legal reviewBiometric laws and platform policies impose consent, retention, and sometimes bansLegal counsel first; prefer detection-only or consented 1:1 designs (Section 5)One global face-match threshold, never tested by groupUneven false accepts/rejects across demographicsValidation set per condition and subgroup; conservative thresholds (Sections 5, 12)Comparing embeddings from different modelsVectors aren't comparableOne model for index and queries; record model versions (Section 6)Unlimited uploads (size, dimensions, format)Decompression bombs, memory exhaustion, cost abuseValidate headers, cap size and dimensions, rate limit, authenticate (Sections 9, 11)Leaving EXIF metadata in stored imagesLeaks GPS location and device detailsStrip metadata before storing or sharing (Section 9)Depending on a cloud feature without checking its lifecycleRetirements and restricted features break productsRead current docs and retirement notices; hide providers behind an interface (Section 7)Running heavy inference on the request thread, unboundedThread starvation and latency spikesBound concurrency; use background jobs for slow work (Section 11)Quick Reference Table
TaskTypical .NET approachKey practiceImage classification (general)Pre-trained ONNX model + ONNX Runtime; or Azure AI Vision tagsMatch preprocessing exactly; softmax; confidence thresholdImage classification (your categories)ML.NET transfer learning, or PyTorch → ONNXRepresentative labeled data; split by source; evaluate per classObject detectionYOLO-style ONNX model; or Azure AI Vision Objects/PeopleLetterbox in, undo letterbox out; confidence + NMSOCR (photos/general)Azure AI Vision Read; Tesseract for localPreprocess; use word confidenceOCR (documents/fields)Azure AI Document Intelligence (layout, prebuilt, custom)Per-field confidence + validation + human reviewFace detectionOpenCV/DNN detector or managed detectionPrefer for blurring/cropping; low riskFace verification/identificationEmbeddings + threshold/vector search, or Azure Face (Limited Access)Legal review, consent, liveness, subgroup evaluationImage embeddingsAzure AI Vision multimodal embeddings; CLIP via ONNXSame model for index and query; cosine similarity; vector DBVisual search / dedupeEmbeddings + vector searchCache by hash; filter by tenant in the queryImage analysis (captions/tags)Azure.AI.Vision.ImageAnalysisRequest only needed features; cache; Entra ID authClassical CV and videoOpenCvSharp (+ runtime package)DisposeMat; BGR vs RGB; frame skippingResize/convert/encodeImageSharp (managed) or SkiaSharpAutoOrient, strip EXIF, cap dimensionsFlexible image understandingMultimodal LLM viaIChatClient+DataContentValidate outputs; watch cost, latency, hallucinationServingASP.NET Core + singletonInferenceSessionBound concurrency; background jobs; validate uploadsQualityTask-specific metrics on a realistic, sliced test setHuman review for low confidence; monitor driftConclusion
Computer vision in .NET is no longer a research detour: a managed API can read an invoice or caption a photo in one call, an ONNX model can detect objects at video rates inside your own process, ML.NET can fine-tune a classifier on a few hundred of your images, embeddings turn "find me something that looks like this" into a vector search, and a multimodal LLM can answer open-ended questions about almost anything you show it. The real skill is choosing among those options deliberately — by accuracy needs, volume, latency, cost, data sensitivity, and the cost of a mistake — and then engineering the unglamorous parts correctly.Those parts are where most projects succeed or fail. Preprocessing must match what the model was trained on — size, channel order, scaling, orientation — or accuracy collapses without any error message. Postprocessing must decode the model's actual output format, apply thresholds and non-maximum suppression, and map coordinates back to the original image. Native resources must be disposed, models loaded once, concurrency bounded, uploads treated as untrusted input, and metadata stripped. Outputs from OCR and multimodal models must be validated against business rules and routed to humans when confidence is low, because plausible-looking wrong answers are the dangerous kind. Face recognition deserves a legal and ethical decision before a technical one; detection-only or carefully consented, locally matched designs are usually the safer path. And above all, evaluate on images that look like production — sliced by condition, class, and (where people appear) demographic group — because a benchmark score on clean photos tells you very little about how the system will behave in the field. Build with those habits and computer vision becomes a reliable component of your .NET applications rather than a demo that stops working the moment real images arrive.
Found this useful? Feel free to star the repo, open an issue with corrections, or share the "the model worked perfectly until we noticed phone photos were all sideways" story that made the case for AutoOrient and representative test data click better than any benchmark.#dotnet #ai #programming #learning #software #coding #development #engineering #inclusive #community
Computer Vision with .NET