Bee Wing Intelligence

Computer vision · Image segmentation · Interpretable ML · PyTorch

Interactive bee species identification from wing imagery

Built by Jiashu Huang jiashuhuang2027@u.northwestern.edu

A portfolio interface for two MLE pipelines: a ResNet-based raw wing classifier and an interpretable skeleton-to-cell geometry model for visual taxonomy. Reviewers can upload a specimen image, run each stage, and open the technical detail.

Input image

Specimen workspace

Ready for step 1
Raw image
Upload a JPG, PNG, or WebP of a single bee wing on an ArUco card.
Single specimen ArUco markers on white background Even lighting Visible wing boundary

Pipeline control

Run stages

    Classification

    Apis mellifera
      98.7% confidence

      Pipeline architecture

      Two paths from field image to species signal

      01 Near-perfect raw image classifier

      Raw image → isolated wing → classification result

      A pre-trained ResNet backbone learns discriminative wing texture and venation features across 20+ species. A paired-wing head can compare latent left/right wing embeddings with Euclidean distance for symmetry-aware health monitoring.

      Technical explanation

      Backend target: accept the uploaded image, remove the ArUco card background while preserving scale metadata, normalize the isolated wing crop, run the ResNet checkpoint, and return top-k species probabilities plus optional left/right latent distance.

      02 Interpretable taxonomy model

      Raw image → wing skeleton → marginal cell → classification result

      A feature extraction pipeline clarifies how visual wing criteria map to taxonomy. It retrieves venation skeletons, isolates the medial/marginal cell shape, applies Fourier decomposition, and classifies species from interpretable geometry.

      Technical explanation

      Backend target: segment the wing, extract the skeleton graph, locate target cells from venation topology, encode cell contours with Fourier descriptors, then run the hierarchical classifier that reached 85% accuracy.

      Project context

      Completed with CCBER at UC Santa Barbara

      This work was completed in collaboration with The Cheadle Center for Biodiversity and Ecological Restoration at UC Santa Barbara during my undergraduate studies. The bee wing image data used for model training and experimentation was provided through CCBER's biodiversity and restoration research collections. I am grateful to the center and its researchers for supporting this project and making the applied taxonomy work possible.

      Technical notes

      How the inference demo is engineered

      Interface orchestration

      Stepwise pipeline state

      The frontend treats each pipeline step as an executable stage with its own endpoint, input contract, artifact dependency, running state, and result renderer. This lets viewers inspect raw input, intermediate artifacts, and final predictions without rerunning completed stages.

      Image normalization

      ArUco-guided wing crop

      OpenCV detects ArUco markers after grayscale conversion, morphological closing, and thresholding. Marker geometry defines a reproducible crop region, preserving scale cues before the image is passed into the deep learning and geometry pipelines.

      ResNet pipeline

      Segmentation plus CNN inference

      A Keras U-Net isolates the wing from the card background, keeps the largest predicted contour, centers the crop around the wing centroid, and resizes it for a fine-tuned PyTorch ResNet50 species classifier. The API returns top-k softmax probabilities.

      Geometry pipeline

      Marginal-cell retrieval

      A ResNet50-encoder segmentation model extracts venation structure. Candidate contours are filtered by area, perimeter ratio, edge contact, and solidity, then a Transformer-based boundary classifier identifies the marginal cell used for shape analysis.

      Shape model

      Fourier descriptor classification

      The marginal-cell contour is traced, resampled to a fixed-length closed boundary, converted into normalized Fourier descriptors, reduced with PCA, and classified with a gradient-boosting model. This path exposes interpretable morphology alongside CNN results.

      Deployment boundary

      Notebook to API service

      Model checkpoints are loaded behind Flask endpoints that exchange uploaded files and generated artifact IDs. The Dockerized backend separates inference from the static frontend, making the demo deployable while keeping the UI responsive to long-running steps.