<![CDATA[Vision-language models such as CLIP achieve strong zero-shot performance but inherit gender bias from their web-scale pretraining data, which is especially visible when the model is used to retrieve images for occupations. Existing prompt-based debiasing methods rely on manually crafted text prompts, which require extensive trial and error and dont transfer easily across professions. This study proposes CoOp with Direction Loss (CoOp+DL), which augments Context Optimization (CoOp), a learnable-prompt method, with an auxiliary loss that pushes the learned prompt representations away from a gender direction computed from contrasting male- and female-referencing prompts. The framework is evaluated on 500 images covering 10 professions with a balanced gender distribution, using three CLIP backbones (ViT-B/32, ViT-B/16, and OpenCLIP ViT-B/32) and three metrics: Gender Bias Score (GBS), Precision-at-K, and SignedSkew. CoOp+DL reduces GBS by 10.3% on ViT-B/32, 5.9% on ViT-B/16, and 9.7% on OpenCLIP, an average of 8.65% across backbones, with bootstrap confidence intervals (n = 1,000) indicating that the direction loss is an active contributor to this reduction rather than an artifact of additional prompt capacity. Retrieval utility (Precision@K) improves on ViT-B/32 and ViT-B/16 (+6.8% and +4.3%) but decreases on OpenCLIP (−8.2%), indicating a backbone-dependent fairness-utility trade-off. CoOp+DL achieves bias reduction that is statistically comparable to a manually engineered ensemble prompt, without requiring manual prompt design. The findings should be interpreted with caution, given the modest evaluation set (500 images, 10 professions) and the binary gender formulation used to define the direction vector, both of which limit generalization and warrant further validation before deployment.]]>
Copyrights © 2026
