Skip to content

Commit af3b449

Browse files
committed
docs: improving README
1 parent c1fdaac commit af3b449

File tree

1 file changed

+294
-40
lines changed

1 file changed

+294
-40
lines changed

README.md

+294-40
Original file line numberDiff line numberDiff line change
@@ -48,8 +48,6 @@ Pydoll is an innovative Python library that's redefining Chromium browser automa
4848
- [Advanced Features](#-advanced-features)
4949
- [Event System](#event-system)
5050
- [Concurrent Scraping](#concurrent-scraping)
51-
- [Proxy Configuration](#proxy-configuration)
52-
- [Troubleshooting](#-troubleshooting)
5351
- [Best Practices](#-best-practices)
5452
- [Contributing](#-contributing)
5553

@@ -82,6 +80,23 @@ async def main():
8280
asyncio.run(main())
8381
```
8482

83+
If you need to configure the browser, you can do it like this:
84+
85+
```python
86+
from pydoll.browser.chrome import Chrome
87+
from pydoll.browser.options import Options
88+
89+
options = Options()
90+
# Public or private proxy, you choose!
91+
options.add_argument('--proxy-server=username:password@ip:port')
92+
# Needs to change the default browser location? No worries!
93+
options.binary_location = '/path/to/your/browser'
94+
async with Chrome(options=options) as browser:
95+
await browser.start()
96+
```
97+
98+
Here you can find all the options available: [Chromium Command Line Switches](https://peter.sh/experiments/chromium-command-line-switches/)
99+
85100
## 🎯 Core Components
86101

87102
### Browser Interface
@@ -517,37 +532,287 @@ await page.execute_script('arguments[0].click()', button)
517532

518533
### WebElement Interface
519534

520-
Interact with elements like a real user:
535+
A WebElement represents a DOM element in the browser, allowing you to interact with it in ways that simulate real human behavior. This section explores the most useful methods for interacting with web elements.
536+
537+
538+
##### `value -> str`
539+
Get the current value of the element - perfect for checking form fields.
521540

522541
```python
523-
async def element_examples():
524-
# Natural and precise interactions
525-
input_field = await page.find_element(By.CSS_SELECTOR, 'input')
526-
await input_field.type_keys('Hello World') # Realistic typing!
527-
528-
# Intuitive chained operations
529-
dropdown = await page.find_element(By.CSS_SELECTOR, 'select')
530-
await dropdown.select_option('value')
542+
# Check the value of an input field
543+
input_element = await page.find_element(By.CSS_SELECTOR, 'input')
544+
print(f"Current value: {input_element.value}")
545+
```
546+
547+
##### `class_name -> str`
548+
Access the element's CSS classes - useful for style verification.
549+
550+
```python
551+
# Check CSS classes
552+
button = await page.find_element(By.CSS_SELECTOR, 'button')
553+
print(f"Classes: {button.class_name}")
554+
```
531555

532-
# Realistic clicks with offset
533-
button = await page.find_element(By.CSS_SELECTOR, 'button')
534-
await button.click(x_offset=5, y_offset=10)
556+
##### `id -> str`
557+
Retrieve the element's ID - perfect for unique identification.
558+
559+
```python
560+
# Get the element's ID
561+
element = await page.find_element(By.CSS_SELECTOR, 'div')
562+
print(f"ID: {element.id}")
563+
```
564+
565+
##### `is_enabled -> bool`
566+
Quickly check if the element is enabled for interaction.
567+
568+
```python
569+
# Check if a button is enabled before clicking
570+
button = await page.find_element(By.CSS_SELECTOR, 'button')
571+
if button.is_enabled:
572+
await button.click()
573+
```
574+
575+
##### `async bounds -> list`
576+
Get the exact coordinates of the element's bounding box.
577+
578+
```python
579+
# Get the element's coordinates
580+
element = await page.find_element(By.CSS_SELECTOR, 'div')
581+
bounds = await element.bounds
582+
print(f"Coordinates: {bounds}")
583+
```
584+
585+
##### `async inner_html -> str`
586+
Retrieve the element's inner HTML - perfect for scraping!
587+
588+
```python
589+
# Get the inner HTML of a container
590+
container = await page.find_element(By.CSS_SELECTOR, '.container')
591+
html = await container.inner_html
592+
print(f"Inner HTML: {html}")
593+
```
594+
595+
##### `async get_element_text() -> str`
596+
Get the text contained in the element - clean and ready to use.
597+
598+
```python
599+
# Get the element's text
600+
element = await page.find_element(By.CSS_SELECTOR, 'p')
601+
text = await element.get_element_text()
602+
print(f"Text: {text}")
603+
```
604+
605+
##### `get_attribute(name: str) -> str`
606+
Access any attribute of the element directly.
607+
608+
```python
609+
# Get a custom attribute value
610+
link = await page.find_element(By.CSS_SELECTOR, 'a')
611+
href = link.get_attribute('href')
612+
data_id = link.get_attribute('data-id')
613+
```
614+
615+
##### `async scroll_into_view()`
616+
Ensure the element is visible before interacting with it.
617+
618+
```python
619+
# Make sure the element is visible
620+
button = await page.find_element(By.CSS_SELECTOR, 'button.footer')
621+
await button.scroll_into_view()
622+
await button.click()
623+
```
624+
625+
##### `async click(x_offset: int = 0, y_offset: int = 0)`
626+
Click elements with amazing precision - even with offsets.
627+
628+
```python
629+
# Click with custom offset
630+
slider = await page.find_element(By.CSS_SELECTOR, '.slider')
631+
# Click 10px to the right of the element's center
632+
await slider.click(x_offset=10, y_offset=0)
633+
```
634+
635+
##### `async click_using_js()`
636+
Click problematic elements that might be obscured or dynamic.
637+
638+
```python
639+
# Use JavaScript to click difficult elements
640+
overlay_button = await page.find_element(By.CSS_SELECTOR, '.overlay-button')
641+
await overlay_button.click_using_js()
642+
```
643+
644+
##### `async send_keys(text: str)`
645+
Send text to form fields quickly.
646+
647+
```python
648+
# Fill a text field
649+
input_field = await page.find_element(By.CSS_SELECTOR, 'input[name="username"]')
650+
await input_field.send_keys("user123")
651+
```
652+
653+
##### `async type_keys(text: str)`
654+
Type realistically, key by key, simulating human input.
655+
656+
```python
657+
# Type like a real human - with pauses between each key
658+
password_field = await page.find_element(By.CSS_SELECTOR, 'input[type="password"]')
659+
await password_field.type_keys("secure_password123") # Realistic typing!
660+
```
661+
662+
##### `async get_screenshot(path: str)`
663+
Capture an image of just the element - perfect for testing or evidence.
664+
665+
```python
666+
# Capture screenshots of specific elements
667+
error_message = await page.find_element(By.CSS_SELECTOR, '.error-message')
668+
await error_message.get_screenshot('/screenshots/error.png')
669+
```
670+
671+
The WebElement interface also inherits element finding capabilities from FindElementsMixin, allowing you to find child elements:
672+
673+
##### `async find_element(by: By, value: str, raise_exc: bool = True)`
674+
Find a child element within this element.
675+
676+
```python
677+
# Find an element within another
678+
container = await page.find_element(By.CSS_SELECTOR, '.container')
679+
button = await container.find_element(By.CSS_SELECTOR, 'button')
680+
await button.click()
681+
```
682+
683+
##### `async find_elements(by: By, value: str, raise_exc: bool = True)`
684+
Find all matching child elements within this element.
685+
686+
```python
687+
# Find all items in a list
688+
list_container = await page.find_element(By.CSS_SELECTOR, 'ul.items')
689+
list_items = await list_container.find_elements(By.CSS_SELECTOR, 'li')
690+
# Iterate through found items
691+
for item in list_items:
692+
text = await item.get_element_text()
693+
print(f"Item: {text}")
694+
```
695+
696+
##### `async wait_element(by: By, value: str, timeout: int = 10, raise_exc: bool = True)`
697+
Wait until a child element appears within this element.
698+
699+
```python
700+
# Wait for an element to appear within another
701+
modal = await page.find_element(By.CSS_SELECTOR, '.modal')
702+
# Wait up to 5 seconds for confirm button to appear in the modal
703+
confirm_button = await modal.wait_element(By.CSS_SELECTOR, '.confirm', timeout=5)
704+
await confirm_button.click()
535705
```
536706

537707
## 🚀 Advanced Features
538708

539709
### Event System
540710

541-
Powerful event system for intelligent automation:
711+
Pydoll's event system is where the magic really happens! Monitor and react to browser events in real-time for incredibly dynamic automation.
712+
713+
##### `async on(event_name: str, callback: callable, temporary: bool = False) -> int`
714+
Register custom callbacks for any browser event. First, enable the events you want to track. Keep in mind that you can use the global or local event listener.
715+
If you want to use the local listener, enable the events on the page instance.
716+
It's really useful, don't you think?
542717

543718
```python
544719
from pydoll.events.page import PageEvents
720+
# Monitors all navigation events on any page
721+
async def on_page_loaded(event):
722+
print(f"🌐 Navigating to: {event['params'].get('url')}")
723+
724+
await browser.enable_page_events() # Activates page events
725+
await browser.on(PageEvents.PAGE_LOADED, on_page_loaded) # Global listener!
726+
# Needs to be locally? Use the page.on method!
727+
await page.on(PageEvents.PAGE_LOADED, on_page_loaded)
728+
```
729+
730+
Need to pass extra parameters to your callback? No problem!
731+
732+
```python
733+
from functools import partial
734+
735+
async def on_page_loaded(page, event):
736+
print(f"📄 Page loaded: {await page.current_url}")
737+
738+
await browser.on(PageEvents.PAGE_LOADED, partial(on_page_loaded, page))
739+
```
740+
741+
##### `async enable_page_events() -> None`
742+
Track everything happening on your pages - loading states, navigation, DOM changes, and more! This works globally or locally. Just use the browser or the page instance to enable the events.
743+
744+
```python
745+
# Enables page event monitoring
746+
await browser.enable_page_events() # global
747+
```
748+
545749

546-
async def event_example():
547-
await page.enable_page_events()
548-
# React to events in real-time!
549-
await page.on(PageEvents.PAGE_LOADED,
550-
lambda e: print('Page loaded successfully!'))
750+
##### `async enable_network_events() -> None`
751+
See all network activity in real-time - perfect for debugging or monitoring specific API calls!
752+
753+
```python
754+
from pydoll.events.network import NetworkEvents
755+
756+
async def on_request(event):
757+
print(f"🔄 Request to: {event['params']['request']['url']} will be sent")
758+
759+
await browser.enable_network_events()
760+
await browser.on(NetworkEvents.REQUEST_WILL_BE_SENT, on_request)
761+
762+
await page.go_to('https://www.google.com') # This will trigger the on_request callback
763+
```
764+
765+
##### `async enable_dom_events() -> None`
766+
Watch the page structure change in real-time and react accordingly!
767+
768+
```python
769+
from pydoll.events.dom import DomEvents
770+
771+
async def on_dom_event(event):
772+
print(f"🔄 The DOM has been updated!")
773+
774+
await browser.enable_dom_events()
775+
await browser.on(DomEvents.DOCUMENT_UPDATED, on_dom_event)
776+
```
777+
778+
##### `async enable_fetch_events(handle_auth_requests: bool = False, resource_type: str = '') -> None`
779+
The ultimate power tool - intercept and modify network requests before they're even sent!
780+
781+
```python
782+
# Intercepts all network requests
783+
from pydoll.events.fetch import FetchEvents
784+
from pydoll.commands.fetch import FetchCommands
785+
from functools import partial
786+
787+
async def interceptor(page, event):
788+
request_id = event['params']['requestId']
789+
request_url = event['params']['request']['url']
790+
print(f"🕵️ Intercepting request to: {request_url}")
791+
792+
# Customize the request however you want!
793+
await page._execute_command(
794+
FetchCommands.continue_request(
795+
request_id=request_id,
796+
method='GET', # change the HTTP method
797+
headers={'X-Custom-Header': 'CustomValue'}, # add your own headers
798+
post_data='Hello World', # modify the request body
799+
url='https://www.google.com', # even change the destination URL!
800+
intercept_response=True # and intercept the response too
801+
)
802+
)
803+
804+
await browser.enable_fetch_events(resource_type='xhr') # only intercept XHR requests
805+
await browser.on(FetchEvents.REQUEST_PAUSED, partial(interceptor, page))
806+
```
807+
808+
With this power, you can transform your automation into something truly intelligent!
809+
810+
##### `async disable_fetch_events() -> None`
811+
Turn off request interception when you're done.
812+
813+
```python
814+
# Disables request interception
815+
await browser.disable_fetch_events()
551816
```
552817

553818
### Concurrent Scraping
@@ -564,27 +829,16 @@ async def concurrent_example():
564829
# Just declare the scrape_page method and see the magic happens!
565830
```
566831

567-
### Proxy Configuration
568-
569-
Robust proxy support, including authentication:
570-
571-
```python
572-
async def proxy_example():
573-
options = Options()
574-
# Private or public proxies, you choose!
575-
options.add_argument('--proxy-server=username:password@ip:port')
576-
577-
async with Chrome(options=options) as browser:
578-
await browser.start()
579-
```
832+
## 🌟 Best Practices
580833

834+
Get the most out of Pydoll with these tips:
581835

582-
For exploring all available methods and additional features, check out:
583-
- Browser interface: [pydoll/browser/base.py](./pydoll/browser/base.py)
584-
- Page interface: [pydoll/browser/page.py](./pydoll/browser/page.py)
585-
- WebElement interface: [pydoll/element.py](./pydoll/element.py)
586-
- Chrome options: [Chromium Command Line Switches](https://peter.sh/experiments/chromium-command-line-switches/)
836+
- Use asynchronous patterns throughout your code for maximum performance
837+
- Prefer specific selectors (IDs, data attributes) over generic ones
838+
- Add proper error handling for robust automation
839+
- Use the event system for reactive scenarios rather than constant polling
840+
- Close browser instances when done to avoid memory leaks
587841

588-
## 🎉 Start Now!
842+
## 🤝 Contributing
589843

590-
Feel free to use, open issues and contributing!
844+
We'd love your help making Pydoll even better! Check out our contribution guidelines to get started. Whether it's fixing bugs, adding features, or improving documentation - all contributions are welcome!

0 commit comments

Comments
 (0)