Building an MCP Browser Automation Tool for Lead Research

We built an internal lead-research workflow that connects browser automation with an MCP service. The goal was to make repetitive research steps easier to run from Claude Code while keeping the session user-controlled and explicit.

Why Build This?

The team needed a faster way to run repeatable lead-research tasks: search for people or companies, collect structured notes, and pass that context into follow-up workflows.

Instead of building a one-off script, we wrapped the workflow as an MCP server. That gave us typed commands, predictable inputs, and a clean interface for assistant-driven tooling.

Architecture

The implementation has three main parts:

A browser automation layer for user-initiated research steps.

A session manager that keeps browser state stable between runs.

An MCP server that exposes the workflow as typed tools.

This kept the assistant-facing interface small while letting the browser workflow remain isolated from the rest of the application.

Session Management

Early versions stored browser profiles in the current working directory. That made sessions inconsistent when the tool was run from different folders.

The fix was to move profile storage to a centralized directory. That made setup more predictable and reduced repeated authentication prompts during normal use.

Interactive Verification

Some sessions require explicit user verification. The CLI handles that by pausing, asking for user input, and resuming the workflow after the verification step is complete.

The important part is that verification remains user-controlled:

python

def _handle_verification_code_challenge(self) -> bool:
    print("\n Email verification code required")
    print("Please check your email for the verification code.")
    verification_code = input("Enter the 6-digit verification code: ").strip()

Structured Extraction

The page structure can differ between account types and views, so the extraction layer uses ordered selectors for each field. This keeps the parser resilient without spreading selector logic across the codebase.

python

def _extract_name_and_url(self, container: WebElement) -> tuple[str, str]:
    name_selectors = [
        'a[data-view-name="search-result-lockup-title"]',  # Free accounts
        'span.entity-result__title-text a',  # Premium variant 1
        'a.app-aware-link span[aria-hidden="true"]',  # Premium variant 2
    ]

Each selector group is owned by a small extraction function. That made failures easier to diagnose and kept future updates localized.

Search Filters

Search filters were separated into a dedicated `SearchFilterHandler` so the main workflow did not need to know how each control is represented in the UI.

Location filters with autocomplete

Industry and company selectors

Current company searches

Connection degree filtering

Follower/connection-of filters

This split also made it easier to test filter behavior independently from result parsing.

The MCP Integration

MCP (Model Context Protocol) by Anthropic allows AI assistants to interact with external data sources.

We exposed the browser workflow as a small set of MCP tools. Claude can request a search, pass structured parameters, and receive normalized results.

With this, Claude can answer queries like:

"Find 10 product managers in San Francisco working at Series B startups."

And here is how the Claude code calls the tool with queries:

text

linkedin-mcp — search_profiles (MCP)(query: "product manager San Francisco Series B", max_results: 10, location: "San Francisco")

Guardrails

This type of tool needs clear operating boundaries. The workflow keeps the user in control of session setup, avoids hidden background activity, and is intended for permissioned research workflows.

What I'd Do Differently

Decouple architecture: Separate driver management from extraction logic earlier.

Async execution: Move to async/await with concurrent drivers for bulk operations.

Error recovery: Add retry logic and exponential backoff for transient errors.

Dynamic selector updates: Build a remote selector update mechanism to avoid code changes.

Current Limitations

Browser workflows are sensitive to UI changes.

Pagination and result limits need explicit handling.

Verification steps still require manual user input.

The tool needs clear usage policies before broader rollout.

Final Thoughts

The useful part of this project was not the browser automation itself. It was the interface around it: typed MCP tools, stable session handling, isolated parsing logic, and explicit user control. Those choices made the workflow easier to reason about and safer to operate.