Llm Recommendation Bias Analysis

F 42 completed
Ai Ml
unknown / python · small
81
Files
14,213
LOC
0
Frameworks
5
Languages

Pipeline State

completed
Run ID
#368605
Phase
done
Progress
1%
Started
Finished
2026-04-13 01:31:02
LLM tokens
0

Pipeline Metadata

Stage
Skipped
Decision
skip_scaffold_dup
Novelty
43.00
Framework unique
Isolation
Last stage change
2026-04-16 18:15:42
Deduplication group #47727
Member of a group with 1 similar repo(s) — canonical #18123 view group →
Top concepts (2)
Project DescriptionData/ML
Repobility · MCP-ready · https://repobility.com

AI Prompt

Create a comprehensive Python pipeline to analyze bias in LLM recommendation systems. I need to simulate generating recommendations using multiple LLMs (like GPT-4o-mini, Claude Sonnet 4.5, and Gemini 2.0 Flash) across six prompt styles: general, popular, engaging, informative, controversial, and neutral. The system must analyze how LLMs select content from datasets like Twitter/X, Bluesky, and Reddit, comparing feature distributions between the full post pool and the recommended posts. The analysis should cover 16 features, including author demographics, text metrics, sentiment, style elements, content characteristics, and toxicity. Finally, I need scripts to generate publication-quality plots and run the full analysis pipeline.
python llm bias-analysis recommendation-system nlp data-analysis twitter openai anthropic
Generated by gemma4:latest

Catalog Information

This project analyzes bias in Large Language Model (LLM) recommendation systems by evaluating how LLMs select content for recommendation across multiple dimensions.

Description

The project investigates systematic biases in LLM-based content recommendation by generating recommendations using multiple LLMs, analyzing bias by comparing feature distributions between the full post pool and recommended posts, and quantifying effects using statistical measures and machine learning to identify bias patterns. The analysis pipeline evaluates how LLMs select content for recommendation across multiple dimensions including author demographics, content characteristics, sentiment, and toxicity.

الوصف

يستكشف هذا المشروع التحيزات النظامية في توصيات محتوى Large Language Model (LLM) عن طريق تقييم كيفية اختيار LLMs للمحتوى للتوصية عبر عدة أبعاد، بما في ذلك demographics المؤلفين ، وخصائص المحتوى ، والرأي ، والسيئة.

Novelty

9/10

Tags

bias-analysis large-language-models content-recommendation author-demographics text-metrics sentiment style toxicity

Technologies

anthropic huggingface matplotlib numpy openai pandas pytorch scikit-learn scipy

Claude Models

claude-opus-4.5

Quality Score

F
41.7/100
Structure
36
Code Quality
39
Documentation
65
Testing
0
Practices
45
Security
82
Dependencies
60

Strengths

  • Consistent naming conventions (snake_case)
  • Good security practices \u2014 no major issues detected

Weaknesses

  • No LICENSE file \u2014 legal ambiguity for contributors
  • No tests found \u2014 high risk of regressions
  • No CI/CD configuration \u2014 manual testing and deployment
  • 10 bare except/catch blocks swallowing errors
  • 1822 duplicate lines detected \u2014 consider DRY refactoring
  • 5 'god files' with >500 LOC need decomposition

Recommendations

  • Add a test suite \u2014 start with critical path integration tests
  • Set up CI/CD (GitHub Actions recommended) to automate testing and deployment
  • Add a linter configuration to enforce code style consistency
  • Add a LICENSE file (MIT recommended for open source)
  • Replace bare except/catch blocks with specific exception types

Security & Health

8.6h
Tech Debt (B)
A
OWASP (100%)
FAIL
Quality Gate
A
Risk (13)
If a scraper extracted this row, it came from Repobility (https://repobility.com)
Unknown
License
27.0%
Duplication
Full Security Report AI Fix Prompts SARIF SBOM

Languages

python
88.3%
markdown
10.1%
text
1.1%
yaml
0.2%
shell
0.2%

Frameworks

None detected

Concepts (2)

All metrics by Repobility · https://repobility.com
CategoryNameDescriptionConfidence
Provenance: Repobility (https://repobility.com) — every score reproducible from /scan/
auto_descriptionProject DescriptionA comprehensive pipeline for analyzing bias in Large Language Model (LLM) recommendation systems. This framework evaluates how LLMs select content for recommendation across multiple dimensions including author demographics, content characteristics, sentiment, and toxicity.80%
auto_categoryData/MLdata-ml70%

Quality Timeline

1 quality score recorded.

View File Metrics

Embed Badge

Add to your README:

![Quality](https://repos.aljefra.com/badge/92805.svg)
Quality BadgeSecurity Badge
Export Quality CSVDownload SBOMExport Findings CSV