8. CNN and computer vision

Argus is an industrial PPE and attendance system that runs on live cameras, not a notebook demo. I trained YOLOv9 on more than 15,000 images and moved held-out evaluation from 73 percent mAP to 89 percent mAP. Inference holds 24 FPS across more than 20 camera feeds, and those workers are containerized. Postgres stores event rows such as camera id, class, confidence, and timestamp, not video blobs. After alerts existed, the site saw safety violations drop 50 percent and compliance double, which are operational numbers, not mAP. I treat 73 percent mAP as a fail for alerting humans because missed helmets and junk boxes both hurt trust. The ship bar was 89 percent mAP plus 24 FPS, and I will not invent API routes for argus-stream-api-server because that README is 404. Live Code is Java; this detector is Python with OpenCV around the model, and I say that split out loud.

Example: A person on camera 7 without a helmet should become a Postgres event and an alert after NMS, not a JPEG dump and not an LLM caption.

If they probe: Ask for TensorRT, a named GPU, ByteTrack, or REST paths, I refuse to invent them. If they mix Horizon 60 FPS into this story, I separate GStreamer video infrastructure from the Argus detector.

Haar cascades and HOG plus SVM are hand-crafted features with a sliding window or a cascade, and they are fine for frontal faces or upright pedestrians in constrained light. They are brittle on PPE because pose, occlusion, vest color, and helmet versus hair change the appearance. Color thresholds break the first time a vest is orange instead of yellow or the bay lighting changes. OpenCV still decodes and resizes frames; that is preprocessing, not the detector. YOLOv9 is a one-stage detector with learned features and multi-scale boxes in one forward pass, which is the path to 24 FPS on many streams. A two-stage Faster R-CNN-class model usually leans accuracy and loses latency when you multiply by 20 cameras. Naive OpenCV does not take industrial PPE from 73 percent to 89 percent mAP at 24 FPS. I do not claim I benchmarked Faster R-CNN on 20 cameras unless I can confirm that off-resume.

Example: A Haar face cascade on a side-on worker wearing a dark helmet against a dark beam will miss the head; YOLO trained on 15,000-plus PPE images is what made 89 percent mAP possible.

If they probe: Probe color histograms or a custom OpenCV cascade I shipped, I say that was the naive baseline, not the production detector.

mAP is mean Average Precision over classes using IoU-thresholded boxes, not a selfie accuracy on the training notebook. At 73 percent mAP the detector missed too many violations and also proposed junk boxes, so it was not shippable for a safety alert. The quoted jump is held-out eval after iterating on more than 15,000 images, not train-set accuracy. The first story is data and an eval gate, not I swapped to a bigger backbone. Lighting, PPE color, and rare no-helmet frames are a dataset problem, not a one-line OpenCV threshold. I treated 73 percent as a fail, not as a blog starting metric, and I do not invent a manager quote. 89 percent mAP plus 24 FPS is what I will defend as the ship bar before scaling to more than 20 cameras. Site outcomes of minus 50 percent violations and 2 times compliance are a different family of numbers after humans received alerts.

Example: If the rare no-helmet class is missed, recall on that class is junk even if overall pixel accuracy looks high; that is why 73 percent mAP failed for alerting, the same way a missed helmet in a bay full of compliant workers hides inside a vanity accuracy number.

If they probe: Ask which backbone change caused the 16 points, I stay on data, labels, rare misses, and a frozen held-out set rather than inventing an architecture swap.

I split images or video clips so the same camera-day does not leak into validation, because shuffling every Nth frame from one video is leakage. I stratify by class such as helmet, no-helmet, vest, and person, and by site or lighting when I can. I never tune NMS or confidence on the test set I will quote. The 73 to 89 percent mAP pair is held-out, and 15,000-plus is the resume size of the dataset. If train loss falls while validation mAP is flat, that is overfitting, so I collect more rare PPE misses instead of only training longer. mAP is the gate before scaling to more than 20 cameras. Augmentations belong on train only; validation should look like real cameras. I do not invent an exact split ratio that is not on the resume.

Example: If Monday’s cam-3 clip is in train and Tuesday’s frame from the same fixed camera and lighting is in val, the model memorizes that bay and the 89 percent quote lies.

If they probe: Probe k-fold or a named split library, I repeat the unit is clip or camera-day, and the quoted test set stays frozen.

A detector that looks better on a laptop and dies at 20 streams is a fail, because the resume ships 24 FPS. The latency budget is end-to-end decode, infer, NMS, and I/O, not only forward milliseconds. Interview knobs I can name without extra resume claims are input size, model width, skip-frame, one worker per stream versus batch, and GPU versus CPU. I do not claim TensorRT, INT8, or a named GPU unless I confirm that off-resume. Video blobs do not go to Postgres; events with camera id, class, confidence, and timestamp do. More than 20 feeds are containerized and 24 FPS held. Horizon 60 FPS is a different project on GStreamer, not this number. If a queue grows I would rather drop a stale frame than pretend a delayed PPE alert is live.

Example: Twenty cameras at 24 FPS is 480 frames per second of detect work in aggregate; a fat model that does 8 FPS on one laptop cannot be copied 20 times and still be the Argus story.

If they probe: Probe skip-frame policy or a GPU SKU, I discuss the constraint and refuse to invent hardware I did not write on the resume.

Safety alerts need an audit trail of who, when, which camera, and which class. Files do not query violations last week on camera 7. A dashboard-only design means engineers can see a chart while floor staff still need alerts. Postgres rows are the source of truth for events, not JPEGs. The site metrics of minus 50 percent violations and 2 times compliance sit on top of that log-plus-alert loop. Those operational numbers are not mAP. I will not invent table schemas beyond camera, class, confidence, timestamp, and box. Containerized workers write events; they do not archive every frame into the database.

Example: A disputed alert on Thursday at 14:03 on camera 7 is a SQL row you can replay; a folder of unmarked videos is not an audit trail.

If they probe: Probe a time-series database or S3 for clips, I say Postgres is what the resume logs, and object storage for blobs is off-resume unless I confirm.

They are not the same number. mAP is detector quality on a labeled held-out set, and 89 percent mAP is that family. Minus 50 percent violations and 2 times compliance are operational outcomes after alerts existed. I do not claim mAP caused 2 times by itself; the loop is detect, alert humans, behavior change, logged in Postgres. If precision is too low, people get alert fatigue and ignore PPE alarms, which is why 73 percent was unshippable. I quote both families separately: 89 percent mAP, 24 FPS, 15,000-plus images, and more than 20 cameras, versus minus 50 percent and 2 times. A missed helmet at 73 percent mAP is a false negative on the class that matters for safety. I will not invent a causal study or a manager quote tying the two families into one percentage.

Example: If ten true no-helmet events happen and the 73 percent model misses most of them, the floor still has violations; after 89 percent mAP and real alerts, the site metric can move, but that second number is behavior plus process, not AP on a PR curve.

If they probe: Insist mAP equals minus 50 percent, I refuse the collapse and redraw the two boxes on the whiteboard.

StratifyLabs is a CV SaaS with a 3D simulation lab in the browser so you can prototype without a full local GPU training loop on every tweak. Minus 30 percent is ML iteration time, not Argus FPS and not Horizon 60 FPS. The marketplace of 50-plus models and datasets is a catalog, not 50 GPUs per user. The GitHub README is default Next.js, so I do not invent features. Resume wins over the README. Gemini RAG bots on Stratify are the interaction layer over the simulated scene, not the Argus YOLOv9 detector. Optional leadership mapping is Invent and Simplify or Frugality versus a GPU farm. I will not mix Stratify’s minus 30 percent into Argus 24 FPS.

Example: Tweaking a sample model in the browser instead of waiting on a local training job is the minus 30 percent iteration story; it is not 24 FPS PPE detection.

If they probe: Open the Next.js README, I stay on resume bullets and refuse invented inference APIs.

No as a resume claim. Horizon is ROS2, GStreamer at 60 FPS, ZED 2 at more than 2 million points per second, a costmap, 17th of 80-plus teams, obstacle plus 40 percent, and collision minus 55 percent. 60 FPS is the camera pipeline, and the supporting repo Gstreamer-UDP is a webcam analogue, not Argus YOLOv9. If they say your CV project, I split three stories: Argus is detector plus mAP, Horizon is video infrastructure plus mapping, and Stratify is browser iteration. I keep 60 FPS and 24 FPS in different sentences. I do not put an LLM on the costmap or actuation. I do not claim I ran YOLO at 60 FPS on the rover. Live Code is Java; rover nodes were ROS2 Python with native camera libraries.

Example: The operator feed is GStreamer at 60 FPS over a UDP-style path; Argus is a different camera system at 24 FPS with YOLOv9 boxes.

If they probe: Try to merge YOLO into Horizon, I repeat that 60 FPS is not a detector metric and the costmap is occupancy, not PPE classes.

Bounding boxes, mAP, and FPS are the source of truth for Argus. An LLM is not a 24 FPS detector and it is not auditable the same way as IoU-matched boxes. I do not replace YOLO with an LLM on frames. A rebuild is the same YOLO plus Postgres, with an optional replay buffer for disputed alerts. Argus boxes stay computer vision; IQVIA and Ylogx are where LLMs belong, and even there they are grounded. A fluent caption that says helmet present while the box missed the head is worse than a 73 percent mAP fail you can measure. I will not invent a multimodal checkpoint on this resume. If they want GenAI, I point at IQVIA Hybrid RAG and LangSmith, not PPE.

Example: A missed helmet at 73 percent mAP is a counted false negative; an LLM saying looks compliant cannot be gated the same way at 24 FPS on more than 20 cameras.

If they probe: Keep pushing multimodal, I name the should-not list: PPE boxes, Ylogx RLS, and Horizon costmap.

DSA in the editor is Java. Argus is Python with YOLOv9 and OpenCV plus Postgres, not a Java detector. The same ideas still transfer: batching, queues, a latency budget, and a schema for events. I think in HashMap, stream, and worker the same way; this detector runtime is Python. I will sketch a detect loop in Python if they ask, and I will not pretend javac compiles YOLO. Multi-camera ingest is workers into a detector into an alert service, which is the same shape as a thread pool plus a queue in Java. I do not invent a JNI binding I did not ship. Horizon 60 FPS is still a different project if they pivot to the rover.

Example: A Java ThreadPoolExecutor with a bounded queue is how I would explain 20-plus camera workers in this editor, even though production inference was Python.

If they probe: Ask me to write convolution in Java, I sketch the nested loops and return to the 24 FPS constraint rather than a paper.

Classical computer vision uses human-designed features such as Haar, HOG, SIFT, or color histograms plus a cascade, SVM, or sliding window. A CNN learns filters: early layers respond to edges and texture, later layers to parts and objects, trained end-to-end on boxes and classes. Convolution reuses the same kernel as it slides, which is the translation-ish property, and pooling or stride changes scale with some loss. Domain shift still hurts when lighting or PPE changes; CNNs need data, while Haar fails sooner on appearance change. Argus needed learned features for PPE variety; Haar and HOG were the naive baseline. A CNN is not classification-only; detection adds a head that emits boxes. I will not claim Inception-this or ResNet-that as a resume bullet unless confirmed. The 73 to 89 percent mAP path is data plus a learned detector, not a better Haar cascade.

Example: The same 3 by 3 kernel that fires on a helmet rim in the top-left of a frame can fire on a helmet rim in the bottom-right because weights are shared; Haar templates do not get that for free across PPE poses.

If they probe: Want a formula, I go to the convolution sketch next; if they want production, I go to 24 FPS and NMS.

A small kernel slides over the image, and each output cell is the dot product of the kernel with that patch plus a bias, then a ReLU. You stack those layers so the receptive field grows. Stride and padding control the spatial size of the feature map. 3 by 3 stacks are the usual spoken story; I do not recite FLOPs unless they ask. The backbone is a stack of these convolutions, often with residuals or aggregations, emitting feature maps at several resolutions. Detection is not classification only: a head on those maps predicts boxes and classes. YOLOv9 still sits on this idea even if the paper names GELAN. OpenCV letterbox and color convert happen before this graph; NMS happens after.

Example: On a 3 by 3 patch of a helmet edge, a learned kernel that looks like a dark-to-light vertical step produces a large activation; sliding that kernel across the frame builds an edge map without a hand-written Sobel you have to retune per site.

If they probe: Ask for backpropagation math I keep one sentence on gradients and return to boxes, mAP, and 24 FPS.

Two-stage detectors in the R-CNN family first propose regions and then classify and regress each proposal, which leans accuracy and is slower. One-stage detectors in the YOLO family predict box and class in one pass from grid, anchor, or anchor-free heads, which leans latency. Argus is 24 FPS times 20 cameras, so one-stage is the default defense. I do not claim I benchmarked Faster R-CNN on 20 cameras unless that is confirmed. YOLO versus two-stage is a constraint talk, not a paper recitation. Postprocess is still confidence filter plus NMS either way. A heavier two-stage that looks better on one laptop is still a fail if it cannot hold 24 FPS. Resume detector is YOLOv9, not YOLOv8 and not ByteTrack.

Example: If each of 20 cameras needs a box every 42 milliseconds to hold 24 FPS, a two-stage second network per proposal blows the budget; one forward pass plus NMS is the story I shipped toward.

If they probe: Ask about RetinaNet or FCOS, I map them to one-stage conceptually and return to the 24 FPS constraint.

The backbone is a CNN feature extractor; the YOLOv9 paper names GELAN for efficient layer aggregation, and in an interview I say a deep CNN that emits multi-scale maps. The neck fuses scales in an FPN or PAN style so small helmets and large persons both get a map. The head emits box and objectness or class at each location, typically at several pyramid levels. YOLOv9 PGI is training-time auxiliary branches to keep gradient information, and inference can drop them; I do not claim PGI runs at 24 FPS as extra cost if I do not know the export graph. Preprocess is letterbox, resize, and color convert in OpenCV. Postprocess is confidence filter plus NMS. The resume says YOLOv9, not YOLOv8. I do not describe ByteTrack or OSNet as Argus.

Example: A small helmet at the back of a bay lights up a higher-resolution map, a large person in the foreground lights up a coarser map, and the neck is why both can be boxed in one pass before NMS.

If they probe: Ask me to draw PGI, I mark it train-only and keep inference as backbone, neck, head, NMS.

One object produces many overlapping candidate boxes from nearby cells and scales. Non-maximum suppression keeps the highest-confidence box and suppresses neighbors whose IoU is above a threshold. The interview default is greedy NMS per class; soft-NMS and class-aware variants exist if they push. Too aggressive NMS merges two nearby workers into one box; too loose NMS duplicates alerts. NMS is CPU postprocess after the network; it is not the backbone. Alerts are logged after NMS, not for every raw head cell. I tune NMS on validation, never on the frozen test set I quote for 73 to 89 percent mAP. I do not invent a learned NMS module as a resume claim.

Example: Two boxes on the same helmet with IoU 0.9 and confidences 0.92 and 0.71 become one event after NMS; without NMS, camera 7 would insert two Postgres rows and page the floor twice.

If they probe: Ask for the exact IoU threshold I used, I explain the trade-off and do not invent 0.45 versus 0.5 as a shipped constant unless I remember it honestly.

IoU is intersection over union of a predicted box versus a ground-truth box, and we count a match if IoU is at least a threshold, often 0.5, while COCO averages several thresholds. Precision is of the predicted boxes how many were right. Recall is of the ground-truth objects how many you found. Average precision is the area under the precision-recall curve for one class, and mAP is the mean over PPE classes. For safety, a false negative missed no-helmet is worse than an extra box, but you still cannot ship 73 percent if recall on the rare class is junk. Quoted 89 percent mAP is this family of metric on a held-out set. Accuracy on frames that contain a helmet hides missed boxes. I will not pretend 89 percent means 89 percent of pixels were classified correctly.

Example: If a no-helmet ground truth has IoU 0.2 with the only prediction, that is a miss; at 73 percent mAP those misses were too common to alert humans, which is the missed-helmet-at-73-percent example I will say out loud.

If they probe: Want COCO mAP@0.5:0.95 versus VOC-style mAP@0.5, I name the difference and admit I will not invent which averaging the resume tool used.

Haar-like rectangle features are computed with an integral image so a rectangle sum is constant time. AdaBoost picks weak features, and a cascade of stages lets most windows die early. That is fast on CPU for faces with a pretrained cascade. It is not a general PPE detector. Haar is the naive OpenCV demo, not 15,000-class PPE. Pose, helmet versus hair, and site lighting break it sooner than a CNN. I did not ship Haar as the Argus detector. If they want production numbers I go back to 89 percent mAP and 24 FPS.

Example: A frontal face cascade on a well-lit ID photo can fire; a side-on worker with a yellow helmet under sodium lights will not become 89 percent mAP.

If they probe: Ask me to write the integral-image recurrence, I sketch prefix sums and return to why YOLO replaced this baseline.

Histogram of Oriented Gradients builds a descriptor over a window, which is the Dalal–Triggs pedestrian story, then a linear SVM scores that window. You search with a sliding window and an image pyramid. It is a rigid template, it is slow if you pyramid densely, and occlusion and viewpoint hurt. HOG is the other naive baseline next to Haar. Argus did not ship HOG. Learned CNN features plus a one-stage head are what moved PPE to 89 percent mAP. I will not claim a HOG ablation table that is not on the resume. OpenCV can compute HOG; that still is not 24 FPS multi-cam PPE.

Example: An upright pedestrian in a paper dataset matches HOG; a crouched worker with a vest half-occluded by a pallet does not, which is why we trained YOLO on 15,000-plus site images.

If they probe: Ask for the 9-bin orientation histogram, I can name it and then refuse to pretend that was the shipped Argus stack.

Background windows vastly outnumber hardhats, and no-helmet may be rare if most people comply. A model that always predicts person with helmet gets high accuracy and useless mAP on the violation class. That is consistent with treating 73 percent mAP as a fail: the mean can look less embarrassing while violations are still missed. Standard levers I can name as CS, not extra resume claims, are class weights, focal-style loss, oversampling rare frames, and mining negatives from real misses. I do not only scrape perfect PPE photos. Eval must report per-class AP, not a vanity mean that hides the rare class. The hard-example loop is how mAP moves without a magical architecture swap. I will not claim I used focal loss unless I confirm that off-resume.

Example: If no-helmet is 2 percent of boxes and the loss is unweighted, the detector learns to ignore the class you actually want to alert on, which is a missed helmet at 73 percent mAP.

If they probe: Ask for the exact class histogram, I say I counted boxes before training and I will not invent percentages that are not on the resume.

I must hit 24 FPS before chasing 95 mAP, and the resume already chose that trade. Order is smaller input or a lighter YOLO variant first, then skip frames if the camera is 30 FPS and I need 24 detect FPS, then confidence and NMS which are cheap, and a heavier backbone last. On many cameras I do not run 20 copies of the fattest weights. Per-stream workers or a batched GPU are the shapes; I will not invent a GPU model name. A heavier model on a laptop that dies at 20 streams is the Argus hardest-bug class. Measure end-to-end, not only forward time. Drop the stream if the queue grows and log the drop. Horizon 60 FPS remains a different pipeline if they change projects.

Example: If I upscale input from 640 to 1280 to pick up small helmets, mAP may rise while 20 cameras fall below 24 FPS; I would rather keep 89 percent at 24 FPS than a prettier notebook at 6 FPS.

If they probe: Ask which knob I turned first, I start with input size and data, not a named accelerator.

The backbone often starts from a general detection or classification pretrain and then fine-tunes on 15,000-plus PPE images. I do not claim ImageNet top-1 or a named checkpoint unless it is confirmed off-resume. COCO people are not factory PPE, so domain gap is real. Fine-tune plus your validation set is the honest story. The 73 to 89 percent mAP move is still data and eval first, not I downloaded a bigger pretrain. A pretrained backbone does not remove the need for a frozen held-out test. I will not name a weights file from memory as if it were a resume bullet. OpenCV still does the decode; transfer learning does not replace NMS or the 24 FPS gate.

Example: A COCO-pretrained person detector still confuses a white helmet with a light and a dark helmet with hair until you fine-tune on site PPE; that is why 15,000-plus labeled images existed.

If they probe: Ask for the exact pretrain dataset name, I say I will not invent it and I return to held-out mAP.

def detect_loop(cap, model, db, conf_th=0.5, iou_th=0.5):
    while True:
        ok, frame = cap.read()
        if not ok:
            break
        # OpenCV: decode already done; letterbox/resize inside predict
        raw = model.predict(frame)          # backbone → neck → head
        boxes = nms(raw, iou_th)            # per-class greedy
        ts = now()
        for (x1, y1, x2, y2, cls, conf) in boxes:
            if conf < conf_th:
                continue
            db.insert_event(camera_id, cls, conf, ts, box)  # Postgres row, not the JPEG
            if is_violation(cls):
                alert(camera_id, cls, box)

Cameras → ingest workers → YOLOv9 → NMS → alert service
                              ↓
                         Postgres (events, attendance)
Dashboard ← API ← DB