開発ツール
trafilatura
Trafilatura — a Python and CLI tool to crawl the web and extract clean text and metadata for corpora and RAG.
★ 6.9k スターApache-2.0
ガイドを見る →// devtools · D4Vinci/Scrapling
Scrapling
Scraplingは、D4Vinciが公開しているオープンソースの開発ツールプロジェクトです（GitHub: D4Vinci/Scrapling）。GitHubスターは86,348、フォークは8,855で、BSD-3-Clauseライセンスの下で公開されています。GitHubでの最終更新は2026年10月8日です。
Adaptive web scraping framework in Python — from a single request to a full-scale crawl
★ 86.3k スター⑂ 8.9k フォークBSD-3-Clause更新 2026年10月
▦ ビジュアル — このプロジェクトのREADMEには埋め込み可能なスクリーンショットがないため、
ここではプレースホルダーを表示しています。下のガイド自体は完全な内容です。
ここではプレースホルダーを表示しています。下のガイド自体は完全な内容です。
// プロジェクトの基本情報
プロジェクト基本情報
- リポジトリ
- D4Vinci/Scrapling
- GitHubスター
- 86,348
- フォーク
- 8,855
- ライセンス
- BSD-3-Clause
- GitHubでの最終更新
- 2026年10月8日
- カテゴリ
- 開発ツール
- 掲載タイプ
- 完全なインストールガイド
- 公式サイト
- Official site
// 仕組み
仕組み
- Clarify your goal
Decide what you want from Scrapling and check the README and requirements to confirm it fits your setup and Python environment.
- Get the code
Clone the repository from GitHub (github.com/D4Vinci/Scrapling) or use the documented install channel if you do not need the source.
- Install and configure
Follow the README quickstart to install dependencies and set any required configuration or credentials.
- Run your first task
Work through the README's first example end to end, then adapt it to your own project or data.
// 機能
選ばれる理由
Adaptive scraping
Selectors adapt when page structure changes, reducing broken scrapers.
Full crawling framework
Includes a Spiders framework for full-scale crawls, not just single-page fetches.
Stealth fetching
Advanced fetching with session support and stealth options built on Playwright.
AI features
Ships AI-oriented extras, including an MCP server and RAG-building support.
Async sessions
Async session management examples for high-throughput scraping.
CLI and shell
Provides a CLI and an interactive shell for quick extraction jobs.
Advertisement
// 自分で実行
自分で実行してみる
- Clone the repository
Start by getting the source and entering the project directory.
git clone https://github.com/D4Vinci/Scrapling.git cd Scrapling - Install Scrapling
Install the core package, or the fetchers extra for browser-based scraping.
pip install scrapling - Install browser dependencies
Required after installing the fetchers extra, per the README.
scrapling install
// 活用例
作れるもの
- Scraping sites whose markup changes frequently
- Building a crawler for price or listing monitoring
- Extracting data behind JavaScript-heavy pages
- Feeding scraped content into a RAG pipeline
Advertisement
// よくある質問
よくある質問
Who maintains Scrapling?
It is maintained by D4Vinci and contributors on GitHub, where the repository github.com/D4Vinci/Scrapling is the canonical home for code, issues and releases.
Is Scrapling free to use?
Yes — it is open source under the BSD-3-Clause license. Read the license in the repository for what is permitted, including any conditions on redistribution or commercial use.
Does it need a paid API key?
No. Scrapling is a local library; optional AI extras use your own model access if you enable them.
How does Scrapling compare to alternatives?
Compared with BeautifulSoup or Scrapy: Scrapling adds adaptive selectors and stealth browser fetching on top of a Scrapy-like crawling model.
How do I get started?
Clone github.com/D4Vinci/Scrapling, follow the README quickstart to install it, and run the documented first example before adapting it to your own work.
// さらに探索
似ているツール
開発ツール
Scrapegraph-ai
Scrape any website with a prompt: LLM-powered scraping pipelines in a few lines of Python
★ 31.6k スターMIT
ガイドを見る →開発ツール
skybridge
Skybridge: the full-stack TypeScript framework for building type-safe MCP Apps and ChatGPT Apps with React.
★ 2.1k スターMIT
ガイドを見る →開発ツール
Gradio
Build machine-learning web demos in Python within minutes — the fastest way to share a model with the world.
★ 43.7k スターApache-2.0
ガイドを見る →開発ツール
ddgs
A Python metasearch library aggregating many web search services — plus CLI, API and MCP server
★ 3.0k スターMIT
ガイドを見る →開発ツール
wigolo
Local-first web search, fetch, crawl and research for AI coding agents over MCP — no API keys.
★ 5.5k スターAGPL-3.0
ガイドを見る →// 代替候補
代替候補： 開発ツール
LibreAIカタログに掲載された開発ツールプロジェクトの中で最もスターの多いもの — Scraplingが合わなかった場合に検討したい、現実的な候補リストです。
- OpenCode★ 211.6k
The open source coding agent
- Transformers★ 166.9k
The model-definition framework behind modern AI — thousands of pretrained transformers for text, vision, and audio.
- claude-code★ 149.7k
Anthropic's agentic coding tool that lives in your terminal and works across your whole codebase
- cc-switch★ 141.3k
Cross-platform desktop all-in-one manager for Claude Code, Codex, OpenCode and other coding agents
- codex★ 128.2k
OpenAI's lightweight coding agent that runs locally in your terminal
// さらに見る devtools
さらに見る： 開発ツール
カタログの中でScraplingとスター数が近い開発ツールプロジェクト — 前後の棚もそのまま見て回れます。
- orca★ 87.8k
The agent development environment for running a fleet of parallel coding agents in worktrees
- llm-course★ 83.3k
A free course to get into large language models, with roadmaps and runnable Colab notebooks
- agent-skills★ 103.3k
Production-grade engineering skills for AI coding agents, by Addy Osmani — 25 skills in one pack
- rtk★ 82.7k
CLI proxy that cuts LLM token consumption 60-90% on common dev commands — one Rust binary
- Gemini Cli★ 107.2k
An open-source AI agent that brings the power of Gemini directly into your terminal
- LlamaFactory★ 75.4k
Unified, efficient fine-tuning of 100+ LLMs and VLMs with a CLI and Web UI — ACL 2024