AI Reviewer Troubleshooting

​

Bot Reviews Not Triggering Workflows

• Bot submits review
• No workflow triggered
• PR not updated

1. Go to bot’s GitHub profile
2. Verify it shows “Bot” badge
3. Check via API: https://api.github.com/users/{bot-username}
4. Look for "type": "Bot" in response

• Use “Update PR” feature
• Manually apply bot feedback
• Trigger workflow explicitly

​

Too Many Bot Workflows

• Bots triggering too many workflows
• High costs
• Excessive automation
• Workflow queue backlog

• Multiple bots reviewing same PR
• Bots reviewing too frequently
• Auto-apply enabled with many bots
• Bots leaving multiple comments

• Temporarily turn off the feature
• Manually review bot feedback
• Selectively apply changes

• Adjust bot settings in repository
• Reduce review frequency
• Filter what bots review
• Limit bot triggers

• Check costs
• Review workflow execution history
• Identify which bots trigger most workflows
• Adjust bot configuration accordingly

Each bot review = 1 workflow execution
1 execution = $0.03
Multiple bots × frequent reviews = high costs

​

Bot Feedback Not Applied Correctly

• Workflow runs but doesn’t fix issues
• Bot re-reviews with same feedback
• Changes don’t match bot requests
• AI misunderstands bot comments

• Bot feedback unclear for AI
• Complex fixes beyond AI capability
• Conflicting feedback from multiple bots
• Bot using uncommon terminology

1. Check what AI agent attempted
2. See if it understood bot feedback
3. Identify where it failed
4. Look for error messages

1. Disable auto-apply temporarily
2. Manually fix based on bot feedback
3. Push updates to PR
4. Re-enable auto-apply after resolution

• Configure bots to give specific guidance
• Use structured feedback formats
• Limit scope of bot reviews
• Ensure bots provide file paths and line numbers

In src/utils/auth.js:45, change:
  if (user == null)
to:
  if (user === null)

Use strict equality for null checks.

Fix the equality checks

​

Bots Not Detected

• Bot has “Bot” badge on GitHub
• Auto-apply enabled
• Reviews still don’t trigger workflows
• Bot appears to be ignored

• Bot account not properly configured
• GitHub API inconsistency
• Webhook not receiving bot reviews
• Bot review format not recognized

1. Check bot is properly installed on repository
2. Verify bot has review permissions
3. Confirm bot is active

1. Have bot submit review
2. Manually use “Update PR” feature
3. Paste bot feedback
4. See if AI can handle it

1. Repository settings → Webhooks
2. Find AutoDev webhook
3. Check for bot review events
4. Verify event payload contains bot info

• Contact AutoDev support
• Provide bot account details
• Share webhook logs
• Request investigation

​

Conflicting Bot Reviews

• Multiple bots review same PR
• Give conflicting feedback
• AI applies one fix, breaks another
• Endless update loop

• Bots have different standards
• One bot’s fix triggers another bot’s review
• Circular dependencies in bot rules
• Incompatible linting rules

1. Turn off auto-apply temporarily
2. Manually reconcile bot feedback
3. Apply fixes that satisfy all bots
4. Re-enable auto-apply

1. Review bot configuration files
2. Ensure linting rules are compatible
3. Disable overlapping checks
4. Use single bot for similar checks

1. Collect all bot feedback
2. Identify conflicts
3. Determine correct approach
4. Manually apply fixes
5. Test with all bots

ESLint: Use single quotes
Prettier: Use double quotes

Solution: Configure both to use same quote style

​

Bot Review Permissions

• Bot cannot submit reviews
• Bot comments appear but not as reviews
• Auto-apply doesn’t recognize bot feedback
• Bot shows access errors

• Bot lacks review permissions
• Bot not invited to repository
• Bot installation incomplete
• Repository permissions too restrictive

1. Repository settings → Collaborators
2. Add bot account if missing
3. Grant “Write” permission minimum
4. Ensure bot can access repository

1. Check app installation
2. Verify repository is included
3. Review app permissions
4. Reinstall app if needed

1. Review bot’s settings
2. Verify repository is enabled
3. Check bot is active
4. Test bot on different PR

​

Webhook Not Receiving Bot Events

• Manual reviews trigger workflows
• Bot reviews don’t
• Webhook logs show user reviews but not bot reviews
• Bot reviews invisible to AutoDev

• Webhook event filters
• GitHub webhook configuration
• Bot reviews sent differently
• Event type mismatch

1. Repository settings → Webhooks
2. Click AutoDev webhook
3. Verify “Pull request reviews” is checked
4. Save webhook configuration

1. Have bot submit review
2. Check webhook Recent Deliveries
3. Look for review event
4. Check event payload

1. Review webhook payload
2. Ensure contains reviewer info
3. Check reviewer type field
4. Verify action is “submitted”

1. Remove AutoDev webhook
2. Reinstall AutoDev GitHub App
3. Webhook will be recreated
4. Test with bot review

​

Rate Limiting Issues

• First bot review works
• Subsequent reviews don’t trigger workflows
• Error messages about rate limits
• Workflows rejected

• GitHub API rate limiting
• AutoDev usage limits
• Too many workflow triggers
• Excessive bot activity

• Wait for rate limit reset (usually 1 hour)
• Reduce bot review frequency
• Stagger bot reviews
• Contact GitHub for higher limits

• Check usage
• Upgrade plan if needed
• Optimize bot usage
• Disable less important bots

Current: 10 bots × 5 reviews/day = 50 workflows
Optimized: 3 bots × 2 reviews/day = 6 workflows

​

Best Practices

​

Next Steps

• AI Reviewer Configuration
• PR Review Process
• Unit Task Troubleshooting