Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement BrowseWeb tool similar to LinearViewIssue tool #930

Draft
wants to merge 2 commits into
base: develop
Choose a base branch
from

Conversation

codegen-sh[bot]
Copy link
Contributor

@codegen-sh codegen-sh bot commented Mar 20, 2025

This PR implements a new BrowseWeb tool that allows Codegen to browse the web, similar to the LinearViewIssue tool.

Changes:

  • Added a new web module in src/codegen/extensions/tools/web/
  • Implemented two main tools:
    • web_search_tool: Searches the web and returns content snippets from search results
    • web_view_page_tool: Views the content of a specific webpage
  • Added necessary dependencies:
    • beautifulsoup4 for HTML parsing
    • responses for testing
  • Added unit tests for the new tools

Implementation Details:

  • The web search tool currently returns mock data, but is designed to be easily extended to use a real search API
  • The web view page tool uses requests and BeautifulSoup to fetch and parse web pages
  • Both tools follow the same pattern as other tools in the codebase, returning Observation objects

Next Steps:

  • Integrate with a real search API (Google Custom Search, Bing Search, etc.)
  • Add more advanced web scraping capabilities
  • Improve error handling and rate limiting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

0 participants