# Table of Contents - [Overview - EdgarTools - Python Library for SEC Data Analysis](#overview-edgartools-python-library-for-sec-data-analysis) - [Overview - EdgarTools - Python Library for SEC Data Analysis](#overview-edgartools-python-library-for-sec-data-analysis) - [Working with Filings - EdgarTools - Python Library for SEC Data Analysis](#working-with-filings-edgartools-python-library-for-sec-data-analysis) - [Overview - EdgarTools - Python Library for SEC Data Analysis](#overview-edgartools-python-library-for-sec-data-analysis) - [Working with Filings - EdgarTools - Python Library for SEC Data Analysis](#working-with-filings-edgartools-python-library-for-sec-data-analysis) - [Installation - EdgarTools - Python Library for SEC Data Analysis](#installation-edgartools-python-library-for-sec-data-analysis) - [Quick Start - EdgarTools - Python Library for SEC Data Analysis](#quick-start-edgartools-python-library-for-sec-data-analysis) - [Python SEC Filings Tutorials | Free Colab Notebooks | EdgarTools - EdgarTools - Python Library for SEC Data Analysis](#python-sec-filings-tutorials-free-colab-notebooks-edgartools-edgartools-python-library-for-sec-data-analysis) - [Overview - EdgarTools - Python Library for SEC Data Analysis](#overview-edgartools-python-library-for-sec-data-analysis) - [MCP Server & Skills - EdgarTools - Python Library for SEC Data Analysis](#mcp-server-skills-edgartools-python-library-for-sec-data-analysis) - [Installation - EdgarTools - Python Library for SEC Data Analysis](#installation-edgartools-python-library-for-sec-data-analysis) - [Company - EdgarTools - Python Library for SEC Data Analysis](#company-edgartools-python-library-for-sec-data-analysis) - [Company - EdgarTools - Python Library for SEC Data Analysis](#company-edgartools-python-library-for-sec-data-analysis) - [Find a Company - EdgarTools - Python Library for SEC Data Analysis](#find-a-company-edgartools-python-library-for-sec-data-analysis) - [Find a Company - EdgarTools - Python Library for SEC Data Analysis](#find-a-company-edgartools-python-library-for-sec-data-analysis) - [Cheat Sheet - EdgarTools - Python Library for SEC Data Analysis](#cheat-sheet-edgartools-python-library-for-sec-data-analysis) - [Choosing the Right API - EdgarTools - Python Library for SEC Data Analysis](#choosing-the-right-api-edgartools-python-library-for-sec-data-analysis) - [MCP Server & Skills - EdgarTools - Python Library for SEC Data Analysis](#mcp-server-skills-edgartools-python-library-for-sec-data-analysis) - [Understanding SEC Filings - EdgarTools - Python Library for SEC Data Analysis](#understanding-sec-filings-edgartools-python-library-for-sec-data-analysis) - [Python SEC Filings Tutorials | Free Colab Notebooks | EdgarTools - EdgarTools - Python Library for SEC Data Analysis](#python-sec-filings-tutorials-free-colab-notebooks-edgartools-edgartools-python-library-for-sec-data-analysis) - [Common Pitfalls - EdgarTools - Python Library for SEC Data Analysis](#common-pitfalls-edgartools-python-library-for-sec-data-analysis) - [Choosing the Right API - EdgarTools - Python Library for SEC Data Analysis](#choosing-the-right-api-edgartools-python-library-for-sec-data-analysis) - [Quick Start - EdgarTools - Python Library for SEC Data Analysis](#quick-start-edgartools-python-library-for-sec-data-analysis) - [Track Insiders - EdgarTools - Python Library for SEC Data Analysis](#track-insiders-edgartools-python-library-for-sec-data-analysis) - [Understanding SEC Filings - EdgarTools - Python Library for SEC Data Analysis](#understanding-sec-filings-edgartools-python-library-for-sec-data-analysis) - [Common Pitfalls - EdgarTools - Python Library for SEC Data Analysis](#common-pitfalls-edgartools-python-library-for-sec-data-analysis) - [Configuration - EdgarTools - Python Library for SEC Data Analysis](#configuration-edgartools-python-library-for-sec-data-analysis) - [Configuration - EdgarTools - Python Library for SEC Data Analysis](#configuration-edgartools-python-library-for-sec-data-analysis) - [The Complete Guide to SEC Filings in Python (2026) - EdgarTools - Python Library for SEC Data Analysis](#the-complete-guide-to-sec-filings-in-python-2026-edgartools-python-library-for-sec-data-analysis) - [The Complete Guide to SEC Filings in Python (2026) - EdgarTools - Python Library for SEC Data Analysis](#the-complete-guide-to-sec-filings-in-python-2026-edgartools-python-library-for-sec-data-analysis) - [Track Insiders - EdgarTools - Python Library for SEC Data Analysis](#track-insiders-edgartools-python-library-for-sec-data-analysis) - [Entity API - EdgarTools - Python Library for SEC Data Analysis](#entity-api-edgartools-python-library-for-sec-data-analysis) - [Examples - EdgarTools - Python Library for SEC Data Analysis](#examples-edgartools-python-library-for-sec-data-analysis) - [Company Subsets - EdgarTools - Python Library for SEC Data Analysis](#company-subsets-edgartools-python-library-for-sec-data-analysis) - [Examples - EdgarTools - Python Library for SEC Data Analysis](#examples-edgartools-python-library-for-sec-data-analysis) - [Attachments - EdgarTools - Python Library for SEC Data Analysis](#attachments-edgartools-python-library-for-sec-data-analysis) - [Attachments - EdgarTools - Python Library for SEC Data Analysis](#attachments-edgartools-python-library-for-sec-data-analysis) - [Cheat Sheet - EdgarTools - Python Library for SEC Data Analysis](#cheat-sheet-edgartools-python-library-for-sec-data-analysis) - [Filter by Criteria - EdgarTools - Python Library for SEC Data Analysis](#filter-by-criteria-edgartools-python-library-for-sec-data-analysis) - [Reference Data - EdgarTools - Python Library for SEC Data Analysis](#reference-data-edgartools-python-library-for-sec-data-analysis) - [Reference Data - EdgarTools - Python Library for SEC Data Analysis](#reference-data-edgartools-python-library-for-sec-data-analysis) - [Overview - EdgarTools - Python Library for SEC Data Analysis](#overview-edgartools-python-library-for-sec-data-analysis) - [Company Classification - EdgarTools - Python Library for SEC Data Analysis](#company-classification-edgartools-python-library-for-sec-data-analysis) - [Business Development Companies - EdgarTools - Python Library for SEC Data Analysis](#business-development-companies-edgartools-python-library-for-sec-data-analysis) - [Entity API - EdgarTools - Python Library for SEC Data Analysis](#entity-api-edgartools-python-library-for-sec-data-analysis) - [Search & Filter - EdgarTools - Python Library for SEC Data Analysis](#search-filter-edgartools-python-library-for-sec-data-analysis) - [Understanding 10-K, 10-Q, and 8-K Report Objects in Python - EdgarTools - Python Library for SEC Data Analysis](#understanding-10-k-10-q-and-8-k-report-objects-in-python-edgartools-python-library-for-sec-data-analysis) - [Business Development Companies - EdgarTools - Python Library for SEC Data Analysis](#business-development-companies-edgartools-python-library-for-sec-data-analysis) - [Insider Trades (Form 4) - EdgarTools - Python Library for SEC Data Analysis](#insider-trades-form-4-edgartools-python-library-for-sec-data-analysis) - [Understanding 10-K, 10-Q, and 8-K Report Objects in Python - EdgarTools - Python Library for SEC Data Analysis](#understanding-10-k-10-q-and-8-k-report-objects-in-python-edgartools-python-library-for-sec-data-analysis) - [Filter by Criteria - EdgarTools - Python Library for SEC Data Analysis](#filter-by-criteria-edgartools-python-library-for-sec-data-analysis) - [Overview - EdgarTools - Python Library for SEC Data Analysis](#overview-edgartools-python-library-for-sec-data-analysis) - [Company Classification - EdgarTools - Python Library for SEC Data Analysis](#company-classification-edgartools-python-library-for-sec-data-analysis) - [Insider Trades (Form 4) - EdgarTools - Python Library for SEC Data Analysis](#insider-trades-form-4-edgartools-python-library-for-sec-data-analysis) - [Search & Filter - EdgarTools - Python Library for SEC Data Analysis](#search-filter-edgartools-python-library-for-sec-data-analysis) - [Beneficial Ownership (13D/G) - EdgarTools - Python Library for SEC Data Analysis](#beneficial-ownership-13d-g-edgartools-python-library-for-sec-data-analysis) - [Prospectus Supplements (424B) - EdgarTools - Python Library for SEC Data Analysis](#prospectus-supplements-424b-edgartools-python-library-for-sec-data-analysis) - [Company Subsets - EdgarTools - Python Library for SEC Data Analysis](#company-subsets-edgartools-python-library-for-sec-data-analysis) - [Beneficial Ownership (13D/G) - EdgarTools - Python Library for SEC Data Analysis](#beneficial-ownership-13d-g-edgartools-python-library-for-sec-data-analysis) - [Track Insider Trading: Analyze SEC Form 4 Buy and Sell Transactions - EdgarTools - Python Library for SEC Data Analysis](#track-insider-trading-analyze-sec-form-4-buy-and-sell-transactions-edgartools-python-library-for-sec-data-analysis) - [Private Offerings (Form D) - EdgarTools - Python Library for SEC Data Analysis](#private-offerings-form-d-edgartools-python-library-for-sec-data-analysis) - [Track Insider Trading: Analyze SEC Form 4 Buy and Sell Transactions - EdgarTools - Python Library for SEC Data Analysis](#track-insider-trading-analyze-sec-form-4-buy-and-sell-transactions-edgartools-python-library-for-sec-data-analysis) - [Private Offerings (Form D) - EdgarTools - Python Library for SEC Data Analysis](#private-offerings-form-d-edgartools-python-library-for-sec-data-analysis) - [Prospectus Supplements (424B) - EdgarTools - Python Library for SEC Data Analysis](#prospectus-supplements-424b-edgartools-python-library-for-sec-data-analysis) - [Query XBRL Facts - EdgarTools - Python Library for SEC Data Analysis](#query-xbrl-facts-edgartools-python-library-for-sec-data-analysis) - [Query XBRL Facts - EdgarTools - Python Library for SEC Data Analysis](#query-xbrl-facts-edgartools-python-library-for-sec-data-analysis) - [Fund Shareholder Reports (N-CSR) - EdgarTools - Python Library for SEC Data Analysis](#fund-shareholder-reports-n-csr-edgartools-python-library-for-sec-data-analysis) - [Fund Portfolios (N-PORT) - EdgarTools - Python Library for SEC Data Analysis](#fund-portfolios-n-port-edgartools-python-library-for-sec-data-analysis) - [Money Market Funds (N-MFP) - EdgarTools - Python Library for SEC Data Analysis](#money-market-funds-n-mfp-edgartools-python-library-for-sec-data-analysis) - [Money Market Funds (N-MFP) - EdgarTools - Python Library for SEC Data Analysis](#money-market-funds-n-mfp-edgartools-python-library-for-sec-data-analysis) - [Fund Census (N-CEN) - EdgarTools - Python Library for SEC Data Analysis](#fund-census-n-cen-edgartools-python-library-for-sec-data-analysis) - [Fund Shareholder Reports (N-CSR) - EdgarTools - Python Library for SEC Data Analysis](#fund-shareholder-reports-n-csr-edgartools-python-library-for-sec-data-analysis) - [Fund Portfolios (N-PORT) - EdgarTools - Python Library for SEC Data Analysis](#fund-portfolios-n-port-edgartools-python-library-for-sec-data-analysis) - [Fund Census (N-CEN) - EdgarTools - Python Library for SEC Data Analysis](#fund-census-n-cen-edgartools-python-library-for-sec-data-analysis) - [Sale Notices (Form 144) - EdgarTools - Python Library for SEC Data Analysis](#sale-notices-form-144-edgartools-python-library-for-sec-data-analysis) - [ABS Distribution (10-D) - EdgarTools - Python Library for SEC Data Analysis](#abs-distribution-10-d-edgartools-python-library-for-sec-data-analysis) - [Sale Notices (Form 144) - EdgarTools - Python Library for SEC Data Analysis](#sale-notices-form-144-edgartools-python-library-for-sec-data-analysis) - [Form Types - EdgarTools - Python Library for SEC Data Analysis](#form-types-edgartools-python-library-for-sec-data-analysis) - [Getting XBRL Data - EdgarTools - Python Library for SEC Data Analysis](#getting-xbrl-data-edgartools-python-library-for-sec-data-analysis) - [Municipal Advisors (MA-I) - EdgarTools - Python Library for SEC Data Analysis](#municipal-advisors-ma-i-edgartools-python-library-for-sec-data-analysis) - [Crowdfunding (Form C) - EdgarTools - Python Library for SEC Data Analysis](#crowdfunding-form-c-edgartools-python-library-for-sec-data-analysis) - [Proxy Statements (DEF 14A) - EdgarTools - Python Library for SEC Data Analysis](#proxy-statements-def-14a-edgartools-python-library-for-sec-data-analysis) - [ABS Distribution (10-D) - EdgarTools - Python Library for SEC Data Analysis](#abs-distribution-10-d-edgartools-python-library-for-sec-data-analysis) - [Crowdfunding (Form C) - EdgarTools - Python Library for SEC Data Analysis](#crowdfunding-form-c-edgartools-python-library-for-sec-data-analysis) - [Municipal Advisors (MA-I) - EdgarTools - Python Library for SEC Data Analysis](#municipal-advisors-ma-i-edgartools-python-library-for-sec-data-analysis) - [Stock Splits & EPS Normalization - EdgarTools - Python Library for SEC Data Analysis](#stock-splits-eps-normalization-edgartools-python-library-for-sec-data-analysis) - [Dimensions - EdgarTools - Python Library for SEC Data Analysis](#dimensions-edgartools-python-library-for-sec-data-analysis) - [Footnotes - EdgarTools - Python Library for SEC Data Analysis](#footnotes-edgartools-python-library-for-sec-data-analysis) - [Footnotes - EdgarTools - Python Library for SEC Data Analysis](#footnotes-edgartools-python-library-for-sec-data-analysis) - [Fund Voting (N-PX) - EdgarTools - Python Library for SEC Data Analysis](#fund-voting-n-px-edgartools-python-library-for-sec-data-analysis) - [Fund Voting (N-PX) - EdgarTools - Python Library for SEC Data Analysis](#fund-voting-n-px-edgartools-python-library-for-sec-data-analysis) - [Form Types - EdgarTools - Python Library for SEC Data Analysis](#form-types-edgartools-python-library-for-sec-data-analysis) - [Getting XBRL Data - EdgarTools - Python Library for SEC Data Analysis](#getting-xbrl-data-edgartools-python-library-for-sec-data-analysis) - [Stock Splits & EPS Normalization - EdgarTools - Python Library for SEC Data Analysis](#stock-splits-eps-normalization-edgartools-python-library-for-sec-data-analysis) - [Dimensions - EdgarTools - Python Library for SEC Data Analysis](#dimensions-edgartools-python-library-for-sec-data-analysis) - [SEC Rate Limits & Compliance - EdgarTools - Python Library for SEC Data Analysis](#sec-rate-limits-compliance-edgartools-python-library-for-sec-data-analysis) - [Proxy Statements (DEF 14A) - EdgarTools - Python Library for SEC Data Analysis](#proxy-statements-def-14a-edgartools-python-library-for-sec-data-analysis) - [SEC Rate Limits & Compliance - EdgarTools - Python Library for SEC Data Analysis](#sec-rate-limits-compliance-edgartools-python-library-for-sec-data-analysis) - [Filing - EdgarTools - Python Library for SEC Data Analysis](#filing-edgartools-python-library-for-sec-data-analysis) - [Local Storage - EdgarTools - Python Library for SEC Data Analysis](#local-storage-edgartools-python-library-for-sec-data-analysis) - [Filing - EdgarTools - Python Library for SEC Data Analysis](#filing-edgartools-python-library-for-sec-data-analysis) - [Local Storage - EdgarTools - Python Library for SEC Data Analysis](#local-storage-edgartools-python-library-for-sec-data-analysis) - [Performance Optimization - EdgarTools - Python Library for SEC Data Analysis](#performance-optimization-edgartools-python-library-for-sec-data-analysis) - [Filings - EdgarTools - Python Library for SEC Data Analysis](#filings-edgartools-python-library-for-sec-data-analysis) - [Filings - EdgarTools - Python Library for SEC Data Analysis](#filings-edgartools-python-library-for-sec-data-analysis) - [Standardization - EdgarTools - Python Library for SEC Data Analysis](#standardization-edgartools-python-library-for-sec-data-analysis) - [Statement Types - EdgarTools - Python Library for SEC Data Analysis](#statement-types-edgartools-python-library-for-sec-data-analysis) - [Period Types - EdgarTools - Python Library for SEC Data Analysis](#period-types-edgartools-python-library-for-sec-data-analysis) - [Period Types - EdgarTools - Python Library for SEC Data Analysis](#period-types-edgartools-python-library-for-sec-data-analysis) - [Current Filings - EdgarTools - Python Library for SEC Data Analysis](#current-filings-edgartools-python-library-for-sec-data-analysis) - [Institutional Holdings (13F) - EdgarTools - Python Library for SEC Data Analysis](#institutional-holdings-13f-edgartools-python-library-for-sec-data-analysis) - [Statement Types - EdgarTools - Python Library for SEC Data Analysis](#statement-types-edgartools-python-library-for-sec-data-analysis) - [Standardization - EdgarTools - Python Library for SEC Data Analysis](#standardization-edgartools-python-library-for-sec-data-analysis) - [XBRL - EdgarTools - Python Library for SEC Data Analysis](#xbrl-edgartools-python-library-for-sec-data-analysis) - [Overview - EdgarTools - Python Library for SEC Data Analysis](#overview-edgartools-python-library-for-sec-data-analysis) - [Institutional Holdings (13F) - EdgarTools - Python Library for SEC Data Analysis](#institutional-holdings-13f-edgartools-python-library-for-sec-data-analysis) - [Overview - EdgarTools - Python Library for SEC Data Analysis](#overview-edgartools-python-library-for-sec-data-analysis) - [Current Filings - EdgarTools - Python Library for SEC Data Analysis](#current-filings-edgartools-python-library-for-sec-data-analysis) - [Cloud Storage (S3) - EdgarTools - Python Library for SEC Data Analysis](#cloud-storage-s3-edgartools-python-library-for-sec-data-analysis) - [XBRL - EdgarTools - Python Library for SEC Data Analysis](#xbrl-edgartools-python-library-for-sec-data-analysis) - [Entity Facts - EdgarTools - Python Library for SEC Data Analysis](#entity-facts-edgartools-python-library-for-sec-data-analysis) - [Cloud Storage (S3) - EdgarTools - Python Library for SEC Data Analysis](#cloud-storage-s3-edgartools-python-library-for-sec-data-analysis) - [Entity Facts - EdgarTools - Python Library for SEC Data Analysis](#entity-facts-edgartools-python-library-for-sec-data-analysis) - [Company Facts - EdgarTools - Python Library for SEC Data Analysis](#company-facts-edgartools-python-library-for-sec-data-analysis) - [Company Facts - EdgarTools - Python Library for SEC Data Analysis](#company-facts-edgartools-python-library-for-sec-data-analysis) - [Multi-Period Analysis - EdgarTools - Python Library for SEC Data Analysis](#multi-period-analysis-edgartools-python-library-for-sec-data-analysis) - [Multi-Period Analysis - EdgarTools - Python Library for SEC Data Analysis](#multi-period-analysis-edgartools-python-library-for-sec-data-analysis) - [Current Events (8-K) - EdgarTools - Python Library for SEC Data Analysis](#current-events-8-k-edgartools-python-library-for-sec-data-analysis) - [Common Issues & Solutions - EdgarTools - Python Library for SEC Data Analysis](#common-issues-solutions-edgartools-python-library-for-sec-data-analysis) - [Common Issues & Solutions - EdgarTools - Python Library for SEC Data Analysis](#common-issues-solutions-edgartools-python-library-for-sec-data-analysis) - [Extract Financial Statements - EdgarTools - Python Library for SEC Data Analysis](#extract-financial-statements-edgartools-python-library-for-sec-data-analysis) - [Extract Financial Statements - EdgarTools - Python Library for SEC Data Analysis](#extract-financial-statements-edgartools-python-library-for-sec-data-analysis) - [Current Events (8-K) - EdgarTools - Python Library for SEC Data Analysis](#current-events-8-k-edgartools-python-library-for-sec-data-analysis) - [Advanced Search - EdgarTools - Python Library for SEC Data Analysis](#advanced-search-edgartools-python-library-for-sec-data-analysis) - [Advanced Search - EdgarTools - Python Library for SEC Data Analysis](#advanced-search-edgartools-python-library-for-sec-data-analysis) - [Customizing Standardization - EdgarTools - Python Library for SEC Data Analysis](#customizing-standardization-edgartools-python-library-for-sec-data-analysis) - [Customizing Standardization - EdgarTools - Python Library for SEC Data Analysis](#customizing-standardization-edgartools-python-library-for-sec-data-analysis) - [Multi-Period Analysis - EdgarTools - Python Library for SEC Data Analysis](#multi-period-analysis-edgartools-python-library-for-sec-data-analysis) - [Multi-Period Analysis - EdgarTools - Python Library for SEC Data Analysis](#multi-period-analysis-edgartools-python-library-for-sec-data-analysis) --- # Overview - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/#edgartools-the-python-library-for-sec-edgar-data) EdgarTools: The Python Library for SEC EDGAR Data ================================================= **Powerful Python library for SEC data analysis and financial research** EdgarTools makes it simple to access, analyze, and extract insights from SEC filings. Whether you're analyzing company financials, tracking insider trading, or researching investment funds, edgartools provides the tools you need. * * * What You Can Do --------------- **Get Key Company Data** Access shares outstanding, public float, and other key data points with simple properties. `company = Company("AAPL") company.shares_outstanding # 15115785000.0 company.public_float # 2899948348000.0` > **[See Apple on edgar.tools — filings, financials, and insider trades in a web UI →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=index) > ** **Analyze Company Financials** Extract financial statements, calculate ratios, and track performance over time. `company = Company("AAPL") financials = company.get_financials() income_statement = financials.income_statement()` **Track Insider Trading** Monitor insider transactions from Forms 3, 4, and 5 with structured data objects. `filings = company.get_filings(form="4").head(10) transactions = pd.concat([f.obj() .to_dataframe() .fillna('') for f in filings])` **Analyze Executive Compensation** Get CEO pay, pay-vs-performance, and 5-year compensation trends from proxy statements. `proxy = company.get_filings(form="DEF 14A").latest().obj() proxy.peo_name # "Mr. Cook" proxy.peo_total_comp # 74609802 proxy.executive_compensation # 5-year DataFrame` **Research Investment Funds** Analyze 13F holdings, track portfolio changes, and compare fund strategies. `fund = Company("BRK-A") holdings = fund.get_filings(form="13F-HR").latest().obj()` **Extract Filing Data** Access any SEC filing since 1994 with clean, structured data extraction. `filing = company.get_filings(form="10-K").latest() text = filing.text() # Clean, readable text` Key Features ------------ ### 🚀 **Easy to Use** * Simple, intuitive API designed for both beginners and experts * Comprehensive documentation with real-world examples * Smart defaults that handle edge cases automatically ### 📊 **Complete SEC Data Access** * **All filing types**: 10-K, 10-Q, 8-K, 13F, Form 4, S-1, and more * **Historical data**: Access filings back to 1994 * **Real-time data**: Get the latest filings as they're published ### 🔍 **Advanced XBRL Support** * Extract structured financial data from XBRL filings * Query individual financial line items with standardized concepts * Handle complex financial statement hierarchies automatically ### ⚡ **Performance Optimized** * Efficient data handling for large datasets * Local caching to minimize API calls * Batch processing capabilities for bulk analysis ### 🛠 **Developer Friendly** * Type hints and comprehensive error handling * Jupyter notebook integration with rich display * Pandas DataFrames for seamless data analysis Installation ------------ Install edgartools with pip: `pip install edgartools` Or use uv for faster installation: `uv pip install edgartools` Get Started in 2 Minutes ------------------------ 1. **Install and set your identity** (required by SEC): `from edgar import * set_identity("your.name@email.com")` 2. **Find a company and get their latest financial data**: `company = Company("TSLA") financials = company.get_financials() financials.income_statement()` See it live on edgar.tools Everything above runs locally. **[edgar.tools](https://app.edgar.tools/?utm_source=edgartools-docs&utm_medium=see-live&utm_content=general) ** puts the same data in a web UI with AI enrichment on top — no code required. * **[Browse Apple's filings, financials, and insider trades →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=general) ** * **[Watch filings arrive in real time →](https://app.edgar.tools/filings?utm_source=edgartools-docs&utm_medium=see-live&utm_content=general) ** * **[Search disclosures across 12 XBRL topics →](https://app.edgar.tools/disclosures?utm_source=edgartools-docs&utm_medium=see-live&utm_content=general) ** Also includes a REST API (20+ endpoints), hosted MCP server, and data exports. Free tier: 100 API calls/day. Popular Use Cases ----------------- ### Financial Analysis * Compare companies across industries * Track financial performance over time * Calculate and analyze financial ratios * Build custom financial dashboards ### Investment Research * Analyze fund holdings and strategy changes * Track insider buying and selling activity * Monitor material events through 8-K filings * Research IPOs and follow-on offerings with parsed prospectus data (price, proceeds, underwriters, dilution) ### Academic Research * Large-scale financial data analysis * Corporate governance studies * Market efficiency research * Regulatory compliance analysis ### AI/ML Applications * Extract clean text for natural language processing * Build predictive models with financial data * Automate document analysis workflows * Create training datasets for financial AI * **Advanced ranking search** with BM25 and semantic structure awareness Why Choose EdgarTools? ---------------------- | Feature | EdgarTools | Alternative Solutions | | --- | --- | --- | | **Ease of Use** | ✅ Simple, Pythonic API | ❌ Complex setup required | | **Data Quality** | ✅ Clean, standardized data | ⚠️ Raw data needs processing | | **Performance** | ✅ Optimized for large datasets | ❌ Slow for bulk operations | | **Documentation** | ✅ Comprehensive with examples | ⚠️ Limited examples | | **Active Development** | ✅ Regular updates and features | ❌ Infrequent updates | | **Community** | ✅ Growing user base | ⚠️ Limited community | Community & Support ------------------- * **📖 Documentation**: Comprehensive guides and API reference * **💬 GitHub Discussions**: Ask questions and share insights * **🐛 Issue Tracker**: Report bugs and request features * **📧 Email Support**: Direct support for enterprise users ### Support the Project If you find EdgarTools useful, please consider supporting its development: [![Buy Me A Coffee](https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png)](https://www.buymeacoffee.com/edgartools) Your support helps maintain and improve EdgarTools for the entire community! What's Next? ------------ **[Quick Start](https://edgartools.readthedocs.io/en/latest/quickstart/) ** - Your first analysis in 5 minutes **[Financial Data](https://edgartools.readthedocs.io/en/latest/guides/financial-data/) ** - Get income statements, balance sheets, cash flow **[Filing Types](https://edgartools.readthedocs.io/en/latest/data-objects/) ** - Work with 10-K, 8-K, 13F, and more **[API Reference](https://edgartools.readthedocs.io/en/latest/api/company/) ** - Complete documentation **[Examples](https://edgartools.readthedocs.io/en/latest/examples/) ** - Real-world code patterns * * * **Ready to start analyzing SEC data?** [Install EdgarTools](https://edgartools.readthedocs.io/en/latest/installation/) and begin your first analysis today. Back to top --- # Overview - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/#edgartools-the-python-library-for-sec-edgar-data) EdgarTools: The Python Library for SEC EDGAR Data ================================================= **Powerful Python library for SEC data analysis and financial research** EdgarTools makes it simple to access, analyze, and extract insights from SEC filings. Whether you're analyzing company financials, tracking insider trading, or researching investment funds, edgartools provides the tools you need. * * * What You Can Do --------------- **Get Key Company Data** Access shares outstanding, public float, and other key data points with simple properties. `company = Company("AAPL") company.shares_outstanding # 15115785000.0 company.public_float # 2899948348000.0` > **[See Apple on edgar.tools — filings, financials, and insider trades in a web UI →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=index) > ** **Analyze Company Financials** Extract financial statements, calculate ratios, and track performance over time. `company = Company("AAPL") financials = company.get_financials() income_statement = financials.income_statement()` **Track Insider Trading** Monitor insider transactions from Forms 3, 4, and 5 with structured data objects. `filings = company.get_filings(form="4").head(10) transactions = pd.concat([f.obj() .to_dataframe() .fillna('') for f in filings])` **Analyze Executive Compensation** Get CEO pay, pay-vs-performance, and 5-year compensation trends from proxy statements. `proxy = company.get_filings(form="DEF 14A").latest().obj() proxy.peo_name # "Mr. Cook" proxy.peo_total_comp # 74609802 proxy.executive_compensation # 5-year DataFrame` **Research Investment Funds** Analyze 13F holdings, track portfolio changes, and compare fund strategies. `fund = Company("BRK-A") holdings = fund.get_filings(form="13F-HR").latest().obj()` **Extract Filing Data** Access any SEC filing since 1994 with clean, structured data extraction. `filing = company.get_filings(form="10-K").latest() text = filing.text() # Clean, readable text` Key Features ------------ ### 🚀 **Easy to Use** * Simple, intuitive API designed for both beginners and experts * Comprehensive documentation with real-world examples * Smart defaults that handle edge cases automatically ### 📊 **Complete SEC Data Access** * **All filing types**: 10-K, 10-Q, 8-K, 13F, Form 4, S-1, and more * **Historical data**: Access filings back to 1994 * **Real-time data**: Get the latest filings as they're published ### 🔍 **Advanced XBRL Support** * Extract structured financial data from XBRL filings * Query individual financial line items with standardized concepts * Handle complex financial statement hierarchies automatically ### ⚡ **Performance Optimized** * Efficient data handling for large datasets * Local caching to minimize API calls * Batch processing capabilities for bulk analysis ### 🛠 **Developer Friendly** * Type hints and comprehensive error handling * Jupyter notebook integration with rich display * Pandas DataFrames for seamless data analysis Installation ------------ Install edgartools with pip: `pip install edgartools` Or use uv for faster installation: `uv pip install edgartools` Get Started in 2 Minutes ------------------------ 1. **Install and set your identity** (required by SEC): `from edgar import * set_identity("your.name@email.com")` 2. **Find a company and get their latest financial data**: `company = Company("TSLA") financials = company.get_financials() financials.income_statement()` See it live on edgar.tools Everything above runs locally. **[edgar.tools](https://app.edgar.tools/?utm_source=edgartools-docs&utm_medium=see-live&utm_content=general) ** puts the same data in a web UI with AI enrichment on top — no code required. * **[Browse Apple's filings, financials, and insider trades →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=general) ** * **[Watch filings arrive in real time →](https://app.edgar.tools/filings?utm_source=edgartools-docs&utm_medium=see-live&utm_content=general) ** * **[Search disclosures across 12 XBRL topics →](https://app.edgar.tools/disclosures?utm_source=edgartools-docs&utm_medium=see-live&utm_content=general) ** Also includes a REST API (20+ endpoints), hosted MCP server, and data exports. Free tier: 100 API calls/day. Popular Use Cases ----------------- ### Financial Analysis * Compare companies across industries * Track financial performance over time * Calculate and analyze financial ratios * Build custom financial dashboards ### Investment Research * Analyze fund holdings and strategy changes * Track insider buying and selling activity * Monitor material events through 8-K filings * Research IPOs and follow-on offerings with parsed prospectus data (price, proceeds, underwriters, dilution) ### Academic Research * Large-scale financial data analysis * Corporate governance studies * Market efficiency research * Regulatory compliance analysis ### AI/ML Applications * Extract clean text for natural language processing * Build predictive models with financial data * Automate document analysis workflows * Create training datasets for financial AI * **Advanced ranking search** with BM25 and semantic structure awareness Why Choose EdgarTools? ---------------------- | Feature | EdgarTools | Alternative Solutions | | --- | --- | --- | | **Ease of Use** | ✅ Simple, Pythonic API | ❌ Complex setup required | | **Data Quality** | ✅ Clean, standardized data | ⚠️ Raw data needs processing | | **Performance** | ✅ Optimized for large datasets | ❌ Slow for bulk operations | | **Documentation** | ✅ Comprehensive with examples | ⚠️ Limited examples | | **Active Development** | ✅ Regular updates and features | ❌ Infrequent updates | | **Community** | ✅ Growing user base | ⚠️ Limited community | Community & Support ------------------- * **📖 Documentation**: Comprehensive guides and API reference * **💬 GitHub Discussions**: Ask questions and share insights * **🐛 Issue Tracker**: Report bugs and request features * **📧 Email Support**: Direct support for enterprise users ### Support the Project If you find EdgarTools useful, please consider supporting its development: [![Buy Me A Coffee](https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png)](https://www.buymeacoffee.com/edgartools) Your support helps maintain and improve EdgarTools for the entire community! What's Next? ------------ **[Quick Start](https://edgartools.readthedocs.io/en/stable/quickstart/) ** - Your first analysis in 5 minutes **[Financial Data](https://edgartools.readthedocs.io/en/stable/guides/financial-data/) ** - Get income statements, balance sheets, cash flow **[Filing Types](https://edgartools.readthedocs.io/en/stable/data-objects/) ** - Work with 10-K, 8-K, 13F, and more **[API Reference](https://edgartools.readthedocs.io/en/stable/api/company/) ** - Complete documentation **[Examples](https://edgartools.readthedocs.io/en/stable/examples/) ** - Real-world code patterns * * * **Ready to start analyzing SEC data?** [Install EdgarTools](https://edgartools.readthedocs.io/en/stable/installation/) and begin your first analysis today. Back to top --- # Working with Filings - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/working-with-filing/#working-with-sec-filings-access-content-attachments-and-structured-data) Working with SEC Filings — Access Content, Attachments, and Structured Data =========================================================================== This guide shows you how to get, view, and extract data from individual SEC filings using the `Filing` class. Getting a Filing ---------------- ### From a Company Get filings for a specific company by ticker or CIK: `from edgar import Company company = Company("AAPL") filings = company.get_filings(form="10-K") filing = filings.latest() print(filing.company) # "Apple Inc." print(filing.form) # "10-K" print(filing.filing_date) # "2023-11-03"` ### From a Filing Search Search across all filings and pick one: `from edgar import get_filings # Get all Q1 2024 10-K filings filings = get_filings(2024, 1, form="10-K") # Get the first one filing = filings[0] # Or get the latest filing = filings.latest()` ### By Accession Number If you know the accession number: `from edgar import get_by_accession_number filing = get_by_accession_number("0000320193-23-000106") print(filing.company) # "Apple Inc."` ### Direct Construction Create a Filing object directly (rarely needed): `from edgar import Filing filing = Filing( form='10-Q', filing_date='2024-06-30', company='Tesla Inc.', cik=1318605, accession_no='0001628280-24-028839' )` Filing Properties ----------------- ### Basic Information `print(f"Company: {filing.company}") print(f"CIK: {filing.cik}") print(f"Form: {filing.form}") print(f"Filing Date: {filing.filing_date}") print(f"Period: {filing.period_of_report}") print(f"Accession: {filing.accession_no}")` **Output:** `Company: Apple Inc. CIK: 320193 Form: 10-K Filing Date: 2023-11-03 Period: 2023-09-30 Accession: 0000320193-23-000106` ### Enhanced Properties (EntityFiling) When you get a filing from a company, you get an `EntityFiling` with additional properties: `company = Company("AAPL") filing = company.get_filings(form="10-K").latest() # EntityFiling-specific properties print(f"Acceptance Time: {filing.acceptance_datetime}") print(f"File Number: {filing.file_number}") print(f"Size: {filing.size} bytes") print(f"Primary Doc: {filing.primary_document}") print(f"Is XBRL: {filing.is_xbrl}") print(f"Is Inline XBRL: {filing.is_inline_xbrl}")` Viewing Filings --------------- ### Open in Browser Open the filing in your default web browser: `filing.open() # Opens primary document` Open the SEC filing homepage (shows all documents): `filing.open_homepage() # Opens index page with all files` ### View in Console Display the filing directly in your terminal or Jupyter notebook: `filing.view()` This uses Rich formatting to display the filing as close to the original as possible. ### Serve Locally Serve the filing on a local HTTP server: `filing.serve(port=8080) # Opens http://localhost:8080 in browser` Accessing Filing Content ------------------------ ### Get HTML `html = filing.html() if html: print(f"HTML length: {len(html)} characters")` ### Get Plain Text `text = filing.text() print(text[:500]) # First 500 characters` ### Get Markdown `md = filing.markdown() # Save to file with open("filing.md", "w") as f: f.write(md)` With page breaks: `md = filing.markdown(include_page_breaks=True, start_page_number=1)` ### Get XML For filings that contain XML (like ownership forms): `xml = filing.xml() if xml: import xml.etree.ElementTree as ET root = ET.fromstring(xml) # Parse XML data` ### Get Full Submission Get the complete SEC text submission file including SGML headers: `full_text = filing.full_text_submission() print(full_text[:1000])` Working with Attachments ------------------------ SEC filings often include multiple documents beyond the primary filing. ### List All Attachments `attachments = filing.attachments print(f"Total attachments: {len(attachments)}") for att in attachments: print(f"{att.sequence}: {att.description}") print(f" Type: {att.document_type}") print(f" File: {att.document}")` ### Get Primary Document `primary = filing.document print(f"Primary: {primary.description}")` ### Access Specific Attachment By index: `first_att = filing.attachments[0]` By document name: `att = filing.attachments["ex-10_1.htm"]` ### Download Attachments Download a specific attachment: `attachment = filing.attachments[0] attachment.download("./downloads/")` Download all attachments: `filing.attachments.download("./downloads/")` ### Work with Exhibits Exhibits are a subset of attachments: `exhibits = filing.exhibits for exhibit in exhibits: print(f"Exhibit {exhibit.exhibit_number}: {exhibit.description}") # Download specific exhibit if exhibit.exhibit_number == "10.1": exhibit.download("./exhibits/")` Extracting Structured Data -------------------------- ### Get XBRL Data For filings with XBRL (10-K, 10-Q, etc.): `xbrl = filing.xbrl() if xbrl: # Access financial statements statements = xbrl.statements income = statements.income_statement() balance = statements.balance_sheet() cashflow = statements.cash_flow_statement() print(income)` ### Preview the Data Object Type Before calling `obj()`, you can check what type of object a filing will return: `filing.obj_type # e.g. 'TenK', 'ThirteenF', 'Form4', None` This is useful for filtering filings to only those with structured data objects. ### Get Form-Specific Object Get a structured object based on the form type: `# For 10-K filing obj = filing.obj() print(type(obj)) # # Access financials from TenK object if obj.financials: income = obj.financials.income_statement balance = obj.financials.balance_sheet print(income)` **Important:** The `financials` property exists on form-specific objects (`TenK`, `TenQ`), not on the base `Filing` class. **Incorrect:** `# This will fail - Filing doesn't have financials financials = filing.financials # AttributeError` **Correct:** `# Get form-specific object first tenk = filing.obj() if tenk.financials: financials = tenk.financials` ### Form-Specific Objects Different forms return different object types: | Form | Object Type | Key Features | | --- | --- | --- | | 10-K | TenK | financials, income\_statement, balance\_sheet, auditor, subsidiaries, reports | | 10-Q | TenQ | financials, income\_statement, balance\_sheet, reports | | 8-K | EightK | items (material events), reports | | 20-F | TwentyF | financials for foreign issuers, reports | | 4 | Form4 | insider transactions | | 13F-HR | ThirteenF | institutional holdings | | SC 13D/G | Schedule13 | beneficial ownership | | DEF 14A | ProxyStatement | proxy voting matters | **Example - 8-K:** `filing = company.get_filings(form="8-K").latest() eightk = filing.obj() print(f"Items: {eightk.items}") # Material events reported` **Example - Form 4:** `filing = company.get_filings(form="4").latest() form4 = filing.obj() # Access insider transaction data html = form4.to_html()` Searching Within a Filing ------------------------- ### Simple Text Search `results = filing.search("artificial intelligence") print(f"Found {len(results)} mentions") for result in results[:3]: print(result[:200]) # First 200 chars of each match` ### Regex Search `# Search for email addresses emails = filing.search( r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b', regex=True ) print(f"Found {len(emails)} email addresses")` ### Search for Financial Terms `# Search for revenue mentions revenue_mentions = filing.search("revenue") # Search for risk factors risk_mentions = filing.search("risk factor") # Case-sensitive regex critical_terms = filing.search(r'\b(material weakness|restatement)\b', regex=True)` Parsing Document Sections ------------------------- ### Get Available Sections `sections = filing.sections() for section in sections: print(section)` ### Parse for Structured Search `doc = filing.parse() # Use parsed document for advanced operations` Saving and Loading ------------------ ### Save Filing `# Save to directory (auto-generates filename) filing.save("./data/filings/") # Save to specific file filing.save("./data/apple_10k_2023.pkl")` ### Load Filing `from edgar import Filing filing = Filing.load("./data/apple_10k_2023.pkl") print(filing.company)` ### Export to Dictionary `data = filing.to_dict() print(data.keys())` ### Export Summary `import pandas as pd summary_df = filing.summary() print(summary_df)` Common Use Cases ---------------- ### Extract Revenue from Latest 10-K `from edgar import Company company = Company("MSFT") filing = company.get_filings(form="10-K").latest() tenk = filing.obj() if tenk.financials: income = tenk.financials.income_statement print(income)` ### Download All Exhibits `filing = get_by_accession_number("0001234567-24-000001") for exhibit in filing.exhibits: print(f"Downloading {exhibit.exhibit_number}: {exhibit.description}") exhibit.download(f"./exhibits/{exhibit.document}")` ### Search Across Recent 8-K Filings `from edgar import get_filings filings = get_filings(2024, 1, form="8-K").head(50) for filing in filings: results = filing.search("earnings") if results: print(f"{filing.company} ({filing.filing_date}): {len(results)} mentions")` ### Convert Filing to Markdown for LLM Analysis `company = Company("NVDA") filing = company.get_filings(form="10-K").latest() # Export to markdown md = filing.markdown(include_page_breaks=True) # Save for processing with open("nvidia_10k_for_analysis.md", "w") as f: f.write(md) print(f"Saved {len(md)} characters to markdown file")` ### Batch Download Filings `from edgar import get_filings filings = get_filings(2023, 4, form="10-K").head(100) for filing in filings: try: filing.download(data_directory="./raw_filings/") print(f"Downloaded: {filing.company}") except Exception as e: print(f"Error downloading {filing.company}: {e}")` Error Handling -------------- Always check for None before using optional data: `from edgar import get_by_accession_number try: filing = get_by_accession_number("0000320193-23-000106") # Check HTML availability html = filing.html() if html is None: print("HTML content not available") else: print(f"HTML: {len(html)} characters") # Check XBRL availability xbrl = filing.xbrl() if xbrl is None: print("No XBRL data available") else: print("XBRL data available") # Check structured object obj = filing.obj() if obj: # Process object pass except Exception as e: print(f"Error processing filing: {e}")` Best Practices -------------- 1. **Check form type** - Use `filing.form` to determine filing type before processing 2. **Verify XBRL** - Check `filing.is_xbrl` for EntityFiling before extracting structured data 3. **Handle large files** - Some filings are very large; consider streaming for attachments 4. **Cache content** - Store downloaded content locally to avoid repeated API calls 5. **Respect rate limits** - Be mindful of SEC rate limits when processing many filings 6. **Use obj() for structure** - Prefer `filing.obj()` over HTML parsing for structured data Performance Tips ---------------- **Efficient pattern:** `# Get form-specific object once obj = filing.obj() # Check before using if obj and hasattr(obj, 'financials') and obj.financials: income = obj.financials.income_statement # Process income statement` **Cache expensive operations:** `# Download once, use multiple times text = filing.text() # Search multiple times without re-downloading results1 = filing.search("revenue") results2 = filing.search("profit")` See Also -------- * **[Filing API Reference](https://edgartools.readthedocs.io/en/latest/api/filing/) ** - Complete Filing class documentation * **[Filings API Reference](https://edgartools.readthedocs.io/en/latest/api/filings/) ** - Working with filing collections * **[Extract Financial Statements](https://edgartools.readthedocs.io/en/latest/guides/extract-statements/) ** - Getting financial data from XBRL * **[Filing Attachments Guide](https://edgartools.readthedocs.io/en/latest/guides/filing-attachments/) ** - Working with documents and exhibits * **[Filtering Filings](https://edgartools.readthedocs.io/en/latest/guides/filtering-filings/) ** - Finding specific filings * **[Current Filings](https://edgartools.readthedocs.io/en/latest/guides/current-filings/) ** - Access today's filings Back to top --- # Overview - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/financial-data/#financial-statements-income-balance-sheet-and-cash-flow-from-sec-filings) Financial Statements: Income, Balance Sheet, and Cash Flow from SEC Filings =========================================================================== **Get financial statements from SEC filings in Python** Quick Start ----------- Three lines to Apple's income statement: `from edgar import Company company = Company("AAPL") financials = company.get_financials() income_statement = financials.income_statement()` ![AAPL Income Statement](https://edgartools.readthedocs.io/en/latest/images/aapl-income-xbrl.webp) That's it. You now have Apple's full income statement from their latest 10-K filing. > **[See Apple's financials on edgar.tools — no code required →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=financial-data) > ** * * * Get Specific Values ------------------- Need just one number? Use the convenience methods: `financials = company.get_financials() revenue = financials.get_revenue() net_income = financials.get_net_income() total_assets = financials.get_total_assets()` | Method | Returns | | --- | --- | | `get_revenue()` | Total revenue / net sales | | `get_net_income()` | Net income | | `get_operating_income()` | Operating income / loss | | `get_total_assets()` | Total assets | | `get_total_liabilities()` | Total liabilities | | `get_stockholders_equity()` | Stockholders' equity | | `get_operating_cash_flow()` | Operating cash flow | | `get_free_cash_flow()` | Free cash flow | | `get_capital_expenditures()` | Capital expenditures | | `get_current_assets()` | Current assets | | `get_current_liabilities()` | Current liabilities | All methods accept `period_offset` to access prior periods (0=current, 1=previous). * * * Available Statements -------------------- `financials = company.get_financials() income = financials.income_statement() balance = financials.balance_sheet() cashflow = financials.cashflow_statement() equity = financials.statement_of_equity() comprehensive = financials.comprehensive_income()` | Statement | Method | | --- | --- | | Income Statement | `income_statement()` | | Balance Sheet | `balance_sheet()` | | Cash Flow Statement | `cashflow_statement()` | | Statement of Equity | `statement_of_equity()` | | Comprehensive Income | `comprehensive_income()` | Cash flow naming The method is `cashflow_statement()` (no underscore between "cash" and "flow"). The `Company` object also accepts `company.cash_flow()` as an alias, but on the `Financials` object always use `cashflow_statement()`. * * * Export to DataFrame ------------------- Every statement converts to a pandas DataFrame: `income = financials.income_statement() # Convert to DataFrame df = income.to_dataframe() # Export to CSV df.to_csv("apple_income_statement.csv") # Export to Excel df.to_excel("apple_income_statement.xlsx")` The DataFrame preserves the statement structure with labeled rows and period columns. * * * Quarterly Financials -------------------- Use `get_quarterly_financials()` for 10-Q data: `quarterly = company.get_quarterly_financials() income = quarterly.income_statement()` ![AAPL Quarterly Income Statement](https://edgartools.readthedocs.io/en/latest/images/aapl-income-quarterly-xbrl.webp) | Need | Method | | --- | --- | | Annual (10-K) | `company.get_financials()` | | Quarterly (10-Q) | `company.get_quarterly_financials()` | * * * Getting Different Levels of Detail ---------------------------------- Financial statements can show different levels of detail. Use the `view` parameter to control what you see: `financials = company.get_financials() # Summary: Matches SEC Viewer (~15-20 rows) income = financials.income_statement(view="summary") # Standard: Matches the filing document (default) income = financials.income_statement(view="standard") # Detailed: All dimensional breakdowns income = financials.income_statement(view="detailed")` ### Summary View ![AAPL Detailed Income Statement](https://edgartools.readthedocs.io/en/latest/images/aapl-income-summary.webp) ### Detailed View ![AAPL Detailed Income Statement](https://edgartools.readthedocs.io/en/latest/images/aapl-income-detailed.webp) | View | Shows | Typical Rows | Best For | | --- | --- | --- | --- | | `"summary"` | Matches SEC Viewer | ~15-20 | Quick overview, validation | | `"standard"` | Matches filing document | ~25-35 | Display, full context | | `"detailed"` | All dimensional breakdowns | ~50+ | Data extraction, segment analysis | ### Example: Apple Revenue With `view="summary"`: `Revenue $391,035M` With `view="standard"`: `Revenue: Products $298,085M Services $92,950M` With `view="detailed"`: `Revenue: Products $298,085M iPhone $201,183M Mac $29,357M iPad $26,694M Wearables, Home and Accessories $40,851M Services $92,950M` ### Views with DataFrames The `view` parameter also works when exporting to DataFrame: `income = financials.income_statement() df_summary = income.to_dataframe(view="summary") df_standard = income.to_dataframe(view="standard") df_detailed = income.to_dataframe(view="detailed")` ### When to Use Each View * **Summary**: Quick checks, matches SEC Viewer, comparing many companies * **Standard**: Matches the filing document, full context with face-level dimensions * **Detailed**: Data extraction, segment analysis, complete dimensional breakdowns **Learn more:** [Dimension Handling Guide](https://edgartools.readthedocs.io/en/latest/xbrl/concepts/dimension-handling/) * * * Compare Multiple Periods ------------------------ To analyze trends across multiple filings, use `XBRLS`: `from edgar.xbrl import XBRLS # Get last 3 annual filings filings = company.get_filings(form="10-K").head(3) # Stitch them together xbrls = XBRLS.from_filings(filings) # Get income statement across all periods income = xbrls.statements.income_statement() # Use view="detailed" for dimensional breakdowns across periods income_detailed = xbrls.statements.income_statement(view="detailed")` The `view` parameter works the same as on single-period statements (`"standard"`, `"detailed"`, `"summary"`). This aligns the periods and concepts across filings for easy comparison. **Learn more:** [Multi-Period Analysis Guide](https://edgartools.readthedocs.io/en/latest/xbrl/guides/multi-period-analysis/) * * * ### Why skip amendments? Use `amendments=False` when fetching filings. Amended filings (10-K/A) sometimes contain only the corrected sections, not complete financial statements. * * * Advanced Topics --------------- ### Raw Facts Query Query individual XBRL facts for research or custom calculations: `xbrl = filing.xbrl() # Find all revenue facts revenue_facts = xbrl.facts.query()\ .by_concept("Revenue")\ .to_dataframe() # Search by label rd_facts = xbrl.facts.query()\ .by_label("Research", exact=False)\ .to_dataframe()` **Learn more:** [XBRL Querying Guide](https://edgartools.readthedocs.io/en/latest/xbrl-querying/) * * * Troubleshooting --------------- ### "No financial data found" Some companies (especially newer or smaller ones) may not have XBRL data: `filing = company.get_filings(form="10-K").latest() if filing.xbrl(): print("XBRL available") else: print("No XBRL - try filing.text() for raw content")` ### "Statement is empty" Try using the detailed view to include all dimensional data: `df = income.to_dataframe(view="detailed")` ### "Numbers don't match the SEC website" Check that you're looking at the right period: `xbrl = filing.xbrl() print(xbrl.reporting_periods)` * * * API Quick Reference ------------------- ### Company-Level (Easiest) | Method | Description | | --- | --- | | `company.get_financials()` | Latest annual financials (10-K) | | `company.get_quarterly_financials()` | Latest quarterly financials (10-Q) | ### Financials Object | Method | Description | | --- | --- | | `financials.income_statement()` | Income statement | | `financials.balance_sheet()` | Balance sheet | | `financials.cashflow_statement()` | Cash flow statement | | `financials.get_revenue()` | Revenue value | | `financials.get_net_income()` | Net income value | | `financials.get_total_assets()` | Total assets value | | `financials.get_financial_metrics()` | Dict of all key metrics | ### Statement Object | Method | Description | | --- | --- | | `statement.to_dataframe()` | Convert to pandas DataFrame | | `statement.to_dataframe(view="summary")` | Matches SEC Viewer | | `statement.to_dataframe(view="standard")` | Matches filing document | | `statement.to_dataframe(view="detailed")` | All dimensional breakdowns | ### Filing-Level (More Control) | Method | Description | | --- | --- | | `filing.xbrl()` | Parse XBRL from filing | | `xbrl.statements.income_statement()` | Get income statement | | `xbrl.facts.query()` | Query individual facts | ### Multi-Period Analysis | Method | Description | | --- | --- | | `XBRLS.from_filings(filings)` | Stitch multiple filings together | | `xbrls.statements.income_statement()` | Aligned multi-period statement | * * * Next Steps ---------- * **[Stock Splits & EPS Normalization](https://edgartools.readthedocs.io/en/latest/guides/stock-splits-eps-normalization/) ** – Detect splits and normalize per-share metrics * **[Multi-Period Analysis](https://edgartools.readthedocs.io/en/latest/xbrl/guides/multi-period-analysis/) ** – Build custom time series * **[Standardization](https://edgartools.readthedocs.io/en/latest/xbrl/concepts/standardization/) ** – How cross-company comparison works * **[XBRL Documentation](https://edgartools.readthedocs.io/en/latest/xbrl/) ** – Complete XBRL reference * * * See it on edgar.tools The code above extracts financials programmatically. **edgar.tools** shows the same data visually — plus Disclosure Search, which lets you browse XBRL topics across every 10-K year for a company. * **[See Apple's financials →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=financial-data) ** * **[Browse income tax disclosures across all filing years →](https://app.edgar.tools/disclosures/income-taxes?utm_source=edgartools-docs&utm_medium=see-live&utm_content=financial-data) ** * **[Explore all 12 XBRL disclosure topics →](https://app.edgar.tools/disclosures?utm_source=edgartools-docs&utm_medium=see-live&utm_content=financial-data) ** Export to Excel, PDF, or CSV with one click. Free tier available. [Pricing →](https://app.edgar.tools/pricing?utm_source=edgartools-docs&utm_medium=see-live&utm_content=financial-data) Back to top --- # Working with Filings - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/working-with-filing/#working-with-sec-filings-access-content-attachments-and-structured-data) Working with SEC Filings — Access Content, Attachments, and Structured Data =========================================================================== This guide shows you how to get, view, and extract data from individual SEC filings using the `Filing` class. Getting a Filing ---------------- ### From a Company Get filings for a specific company by ticker or CIK: `from edgar import Company company = Company("AAPL") filings = company.get_filings(form="10-K") filing = filings.latest() print(filing.company) # "Apple Inc." print(filing.form) # "10-K" print(filing.filing_date) # "2023-11-03"` ### From a Filing Search Search across all filings and pick one: `from edgar import get_filings # Get all Q1 2024 10-K filings filings = get_filings(2024, 1, form="10-K") # Get the first one filing = filings[0] # Or get the latest filing = filings.latest()` ### By Accession Number If you know the accession number: `from edgar import get_by_accession_number filing = get_by_accession_number("0000320193-23-000106") print(filing.company) # "Apple Inc."` ### Direct Construction Create a Filing object directly (rarely needed): `from edgar import Filing filing = Filing( form='10-Q', filing_date='2024-06-30', company='Tesla Inc.', cik=1318605, accession_no='0001628280-24-028839' )` Filing Properties ----------------- ### Basic Information `print(f"Company: {filing.company}") print(f"CIK: {filing.cik}") print(f"Form: {filing.form}") print(f"Filing Date: {filing.filing_date}") print(f"Period: {filing.period_of_report}") print(f"Accession: {filing.accession_no}")` **Output:** `Company: Apple Inc. CIK: 320193 Form: 10-K Filing Date: 2023-11-03 Period: 2023-09-30 Accession: 0000320193-23-000106` ### Enhanced Properties (EntityFiling) When you get a filing from a company, you get an `EntityFiling` with additional properties: `company = Company("AAPL") filing = company.get_filings(form="10-K").latest() # EntityFiling-specific properties print(f"Acceptance Time: {filing.acceptance_datetime}") print(f"File Number: {filing.file_number}") print(f"Size: {filing.size} bytes") print(f"Primary Doc: {filing.primary_document}") print(f"Is XBRL: {filing.is_xbrl}") print(f"Is Inline XBRL: {filing.is_inline_xbrl}")` Viewing Filings --------------- ### Open in Browser Open the filing in your default web browser: `filing.open() # Opens primary document` Open the SEC filing homepage (shows all documents): `filing.open_homepage() # Opens index page with all files` ### View in Console Display the filing directly in your terminal or Jupyter notebook: `filing.view()` This uses Rich formatting to display the filing as close to the original as possible. ### Serve Locally Serve the filing on a local HTTP server: `filing.serve(port=8080) # Opens http://localhost:8080 in browser` Accessing Filing Content ------------------------ ### Get HTML `html = filing.html() if html: print(f"HTML length: {len(html)} characters")` ### Get Plain Text `text = filing.text() print(text[:500]) # First 500 characters` ### Get Markdown `md = filing.markdown() # Save to file with open("filing.md", "w") as f: f.write(md)` With page breaks: `md = filing.markdown(include_page_breaks=True, start_page_number=1)` ### Get XML For filings that contain XML (like ownership forms): `xml = filing.xml() if xml: import xml.etree.ElementTree as ET root = ET.fromstring(xml) # Parse XML data` ### Get Full Submission Get the complete SEC text submission file including SGML headers: `full_text = filing.full_text_submission() print(full_text[:1000])` Working with Attachments ------------------------ SEC filings often include multiple documents beyond the primary filing. ### List All Attachments `attachments = filing.attachments print(f"Total attachments: {len(attachments)}") for att in attachments: print(f"{att.sequence}: {att.description}") print(f" Type: {att.document_type}") print(f" File: {att.document}")` ### Get Primary Document `primary = filing.document print(f"Primary: {primary.description}")` ### Access Specific Attachment By index: `first_att = filing.attachments[0]` By document name: `att = filing.attachments["ex-10_1.htm"]` ### Download Attachments Download a specific attachment: `attachment = filing.attachments[0] attachment.download("./downloads/")` Download all attachments: `filing.attachments.download("./downloads/")` ### Work with Exhibits Exhibits are a subset of attachments: `exhibits = filing.exhibits for exhibit in exhibits: print(f"Exhibit {exhibit.exhibit_number}: {exhibit.description}") # Download specific exhibit if exhibit.exhibit_number == "10.1": exhibit.download("./exhibits/")` Extracting Structured Data -------------------------- ### Get XBRL Data For filings with XBRL (10-K, 10-Q, etc.): `xbrl = filing.xbrl() if xbrl: # Access financial statements statements = xbrl.statements income = statements.income_statement() balance = statements.balance_sheet() cashflow = statements.cash_flow_statement() print(income)` ### Preview the Data Object Type Before calling `obj()`, you can check what type of object a filing will return: `filing.obj_type # e.g. 'TenK', 'ThirteenF', 'Form4', None` This is useful for filtering filings to only those with structured data objects. ### Get Form-Specific Object Get a structured object based on the form type: `# For 10-K filing obj = filing.obj() print(type(obj)) # # Access financials from TenK object if obj.financials: income = obj.financials.income_statement balance = obj.financials.balance_sheet print(income)` **Important:** The `financials` property exists on form-specific objects (`TenK`, `TenQ`), not on the base `Filing` class. **Incorrect:** `# This will fail - Filing doesn't have financials financials = filing.financials # AttributeError` **Correct:** `# Get form-specific object first tenk = filing.obj() if tenk.financials: financials = tenk.financials` ### Form-Specific Objects Different forms return different object types: | Form | Object Type | Key Features | | --- | --- | --- | | 10-K | TenK | financials, income\_statement, balance\_sheet, auditor, subsidiaries, reports | | 10-Q | TenQ | financials, income\_statement, balance\_sheet, reports | | 8-K | EightK | items (material events), reports | | 20-F | TwentyF | financials for foreign issuers, reports | | 4 | Form4 | insider transactions | | 13F-HR | ThirteenF | institutional holdings | | SC 13D/G | Schedule13 | beneficial ownership | | DEF 14A | ProxyStatement | proxy voting matters | **Example - 8-K:** `filing = company.get_filings(form="8-K").latest() eightk = filing.obj() print(f"Items: {eightk.items}") # Material events reported` **Example - Form 4:** `filing = company.get_filings(form="4").latest() form4 = filing.obj() # Access insider transaction data html = form4.to_html()` Searching Within a Filing ------------------------- ### Simple Text Search `results = filing.search("artificial intelligence") print(f"Found {len(results)} mentions") for result in results[:3]: print(result[:200]) # First 200 chars of each match` ### Regex Search `# Search for email addresses emails = filing.search( r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b', regex=True ) print(f"Found {len(emails)} email addresses")` ### Search for Financial Terms `# Search for revenue mentions revenue_mentions = filing.search("revenue") # Search for risk factors risk_mentions = filing.search("risk factor") # Case-sensitive regex critical_terms = filing.search(r'\b(material weakness|restatement)\b', regex=True)` Parsing Document Sections ------------------------- ### Get Available Sections `sections = filing.sections() for section in sections: print(section)` ### Parse for Structured Search `doc = filing.parse() # Use parsed document for advanced operations` Saving and Loading ------------------ ### Save Filing `# Save to directory (auto-generates filename) filing.save("./data/filings/") # Save to specific file filing.save("./data/apple_10k_2023.pkl")` ### Load Filing `from edgar import Filing filing = Filing.load("./data/apple_10k_2023.pkl") print(filing.company)` ### Export to Dictionary `data = filing.to_dict() print(data.keys())` ### Export Summary `import pandas as pd summary_df = filing.summary() print(summary_df)` Common Use Cases ---------------- ### Extract Revenue from Latest 10-K `from edgar import Company company = Company("MSFT") filing = company.get_filings(form="10-K").latest() tenk = filing.obj() if tenk.financials: income = tenk.financials.income_statement print(income)` ### Download All Exhibits `filing = get_by_accession_number("0001234567-24-000001") for exhibit in filing.exhibits: print(f"Downloading {exhibit.exhibit_number}: {exhibit.description}") exhibit.download(f"./exhibits/{exhibit.document}")` ### Search Across Recent 8-K Filings `from edgar import get_filings filings = get_filings(2024, 1, form="8-K").head(50) for filing in filings: results = filing.search("earnings") if results: print(f"{filing.company} ({filing.filing_date}): {len(results)} mentions")` ### Convert Filing to Markdown for LLM Analysis `company = Company("NVDA") filing = company.get_filings(form="10-K").latest() # Export to markdown md = filing.markdown(include_page_breaks=True) # Save for processing with open("nvidia_10k_for_analysis.md", "w") as f: f.write(md) print(f"Saved {len(md)} characters to markdown file")` ### Batch Download Filings `from edgar import get_filings filings = get_filings(2023, 4, form="10-K").head(100) for filing in filings: try: filing.download(data_directory="./raw_filings/") print(f"Downloaded: {filing.company}") except Exception as e: print(f"Error downloading {filing.company}: {e}")` Error Handling -------------- Always check for None before using optional data: `from edgar import get_by_accession_number try: filing = get_by_accession_number("0000320193-23-000106") # Check HTML availability html = filing.html() if html is None: print("HTML content not available") else: print(f"HTML: {len(html)} characters") # Check XBRL availability xbrl = filing.xbrl() if xbrl is None: print("No XBRL data available") else: print("XBRL data available") # Check structured object obj = filing.obj() if obj: # Process object pass except Exception as e: print(f"Error processing filing: {e}")` Best Practices -------------- 1. **Check form type** - Use `filing.form` to determine filing type before processing 2. **Verify XBRL** - Check `filing.is_xbrl` for EntityFiling before extracting structured data 3. **Handle large files** - Some filings are very large; consider streaming for attachments 4. **Cache content** - Store downloaded content locally to avoid repeated API calls 5. **Respect rate limits** - Be mindful of SEC rate limits when processing many filings 6. **Use obj() for structure** - Prefer `filing.obj()` over HTML parsing for structured data Performance Tips ---------------- **Efficient pattern:** `# Get form-specific object once obj = filing.obj() # Check before using if obj and hasattr(obj, 'financials') and obj.financials: income = obj.financials.income_statement # Process income statement` **Cache expensive operations:** `# Download once, use multiple times text = filing.text() # Search multiple times without re-downloading results1 = filing.search("revenue") results2 = filing.search("profit")` See Also -------- * **[Filing API Reference](https://edgartools.readthedocs.io/en/stable/api/filing/) ** - Complete Filing class documentation * **[Filings API Reference](https://edgartools.readthedocs.io/en/stable/api/filings/) ** - Working with filing collections * **[Extract Financial Statements](https://edgartools.readthedocs.io/en/stable/guides/extract-statements/) ** - Getting financial data from XBRL * **[Filing Attachments Guide](https://edgartools.readthedocs.io/en/stable/guides/filing-attachments/) ** - Working with documents and exhibits * **[Filtering Filings](https://edgartools.readthedocs.io/en/stable/guides/filtering-filings/) ** - Finding specific filings * **[Current Filings](https://edgartools.readthedocs.io/en/stable/guides/current-filings/) ** - Access today's filings Back to top --- # Installation - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/installation/#installation) Installation ============ Get started with edgartools in minutes. This guide covers all installation methods and system requirements. System Requirements ------------------- * **Python**: 3.8 or higher Quick Installation ------------------ ### Using pip (Recommended) `pip install edgartools` For the latest features and bug fixes: `pip install -U edgartools` ### Using uv (Fast Alternative) `uv pip install edgartools` Development Installation ------------------------ If you want to contribute or use the latest development version: `# Clone the repository git clone https://github.com/dgunning/edgartools.git cd edgartools # Install in development mode pip install -e . # Or with development dependencies pip install -e ".[dev]"` Verify Installation ------------------- Test your installation by running this simple command: `from edgar import get_filings print("EdgarTools installed successfully!")` Expected output: `EdgarTools installed successfully!` If you see this message, your installation is successful. If you see `ImportError: cannot import name 'get_filings' from 'edgar'` then you have likely installed another package named **edgar** not **edgartools**. If you encounter this error, uninstall the conflicting package and reinstall edgartools: `pip uninstall edgar pip install edgartools` Setting Your Identity --------------------- Before using edgartools, you must set your identity to comply with SEC requirements: ### Method 1: In Python Code `from edgar import set_identity # Use your name and email set_identity("John Doe john.doe@company.com") # Or just your email set_identity("john.doe@company.com")` ### Method 2: Environment Variable Set the `EDGAR_IDENTITY` environment variable: **Linux/macOS:** `export EDGAR_IDENTITY="John Doe john.doe@company.com"` **Windows:** `set EDGAR_IDENTITY=John Doe john.doe@company.com` **Windows PowerShell:** `$env:EDGAR_IDENTITY = "John Doe john.doe@company.com"` Optional Dependencies --------------------- For enhanced functionality, install these optional packages: Troubleshooting --------------- ### Common Issues #### ImportError: No module named 'edgar' **Problem**: Package not installed correctly **Solution**: `pip uninstall edgar pip install --force-reinstall edgartools` #### SEC Identity Error **Problem**: Identity not set **Solution**: Follow the [Setting Your Identity](https://edgartools.readthedocs.io/en/latest/installation/#setting-your-identity) section above #### Permission Errors on Windows **Problem**: Insufficient permissions **Solution**: Run as administrator or use `--user` flag: `pip install --user edgartools` #### SSL Certificate Errors **Problem**: Corporate firewall or proxy **Solution**: Configure pip for your proxy: `pip install --trusted-host pypi.org --trusted-host pypi.python.org edgartools` #### Memory Issues with Large Datasets **Problem**: Out of memory errors **Solution**: - Increase system memory - Use data chunking techniques - Process data in smaller batches ### Getting Help If you encounter issues: 1. **Search existing issues**: [GitHub Issues](https://github.com/dgunning/edgartools/issues) 2. **Create a new issue**: Include Python version, OS, and error messages 3. **Join the community**: Discussions and support channels Virtual Environment Setup ------------------------- For isolated development, use virtual environments: ### Using venv (Python 3.8+) `# Create virtual environment python -m venv edgar-env # Activate (Linux/macOS) source edgar-env/bin/activate # Activate (Windows) edgar-env\Scripts\activate # Install edgartools pip install edgartools # Deactivate when done deactivate` Performance Optimization ------------------------ For optimal performance: 1. **Use Local Storage** to download and work with SEC filings locally 2. **Set reasonable limits** when querying large datasets 3. **Use filtering** to reduce data transfer Next Steps ---------- After installation: 1. **Read the [Quick Start Guide](https://edgartools.readthedocs.io/en/latest/quickstart/) ** for your first analysis 2. **Check the [API Reference](https://edgartools.readthedocs.io/en/latest/api/company/) ** for detailed documentation Security Considerations ----------------------- * **Never commit your identity** to version control * **Use environment variables** for production deployments * **Follow SEC rate limits** to avoid being blocked * **Keep your installation updated** for security patches License ------- EdgarTools is released under the MIT License. See [LICENSE](https://github.com/dgunning/edgartools/blob/main/LICENSE) for details. Back to top --- # Quick Start - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/quickstart/#quick-start-guide) Quick Start Guide ================= Get up and running with EdgarTools in 5 minutes. By the end, you'll have a company's financial statements in Python. Prerequisites ------------- * Python 3.8 or higher * Internet connection * Basic familiarity with Python Step 1: Install EdgarTools -------------------------- `pip install edgartools` Trouble importing? If you see `ImportError: cannot import name 'get_filings' from 'edgar'`, you may have installed the wrong package. There is an unrelated package called `edgar` on PyPI. Fix it with: `pip uninstall edgar && pip install edgartools` Step 2: Set Your Identity ------------------------- The SEC requires all API users to identify themselves. Set your identity once: `from edgar import set_identity # Use your name and email (required by SEC) set_identity("John Doe john.doe@company.com")` **Tip:** You can also set the `EDGAR_IDENTITY` environment variable to avoid doing this in every script. Step 3: Get a Company --------------------- Look up any public company by ticker symbol or CIK number: `from edgar import Company company = Company("AAPL") # Apple Inc.` ![AAPL](https://edgartools.readthedocs.io/en/latest/images/AAPL.png) > **[See Apple's filings, financials, and insider trades on edgar.tools — no code required →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=quickstart) > ** You can access basic company data as properties: `company.industry # 'ELECTRONIC COMPUTERS' company.shares_outstanding # 15115785000.0 company.public_float # 2899948348000.0` Step 4: Get Financial Statements -------------------------------- This is the most common task — getting a company's financial statements: `financials = company.get_financials() # The three financial statements income = financials.income_statement() balance = financials.balance_sheet() cashflow = financials.cashflow_statement()` ![AAPL Income Statement](https://edgartools.readthedocs.io/en/latest/images/aapl-income-xbrl.webp) That's it — three lines to get any company's income statement, balance sheet, or cash flow. Common gotcha The canonical method is `cashflow_statement()`, but `cash_flow_statement()` also works. All three statements: `income_statement()`, `balance_sheet()`, `cashflow_statement()`. Step 5: Get Specific Values --------------------------- Need just one number instead of the full statement? `financials.get_revenue() # 391035000000 financials.get_net_income() # 93736000000` Step 6: Export to DataFrame --------------------------- Every financial statement converts to a pandas DataFrame for further analysis: `df = financials.income_statement().to_dataframe()` You can also export company filings: `filings = company.get_filings() df = filings.to_pandas()` Step 7: Browse Company Filings ------------------------------ Retrieve and filter a company's SEC filings: `# Get all filings filings = company.get_filings() # Filter by form type tenk_filings = company.get_filings(form="10-K") # Get the latest 10-K as a data object tenk = company.latest("10-K")` ![AAPL Filings](https://edgartools.readthedocs.io/en/latest/images/aapl-filings.png) How EdgarTools Is Organized --------------------------- Here's a map of the main objects. Use it as a reference when you want to try something new: `Company("AAPL") # Start here — look up a company ├── .get_financials() # Annual financials from 10-K (RECOMMENDED) │ ├── .income_statement() # Revenue, expenses, profit │ ├── .balance_sheet() # Assets, liabilities, equity │ ├── .cashflow_statement() # Cash in and out │ ├── .get_revenue() # Quick: just the revenue number │ └── .get_net_income() # Quick: just net income │ ├── .get_quarterly_financials() # Quarterly financials from 10-Q │ └── (same interface as above) │ ├── .get_filings(form="10-K") # Browse SEC filings │ ├── .head(5) # See the first 5 │ ├── .latest() # Get the most recent one │ └── [0].obj() # Parse into a data object (TenK, etc.) │ ├── .auditor # Auditor name, PCAOB ID, location │ └── .subsidiaries # Subsidiaries from Exhibit 21 │ └── .get_facts() # Historical data (for 4+ years of trends) ├── .income_statement() # Multi-year income data └── .balance_sheet() # Multi-year balance sheet` See it live on edgar.tools The code above runs locally. **[edgar.tools](https://app.edgar.tools/?utm_source=edgartools-docs&utm_medium=see-live&utm_content=general) ** puts the same data in a web UI with AI enrichment on top — no code required. * **[Browse Apple's filings, financials, and insider trades →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=general) ** * **[Watch filings arrive in real time →](https://app.edgar.tools/filings?utm_source=edgartools-docs&utm_medium=see-live&utm_content=general) ** * **[Search disclosures across 12 XBRL topics →](https://app.edgar.tools/disclosures?utm_source=edgartools-docs&utm_medium=see-live&utm_content=general) ** Also includes a REST API (20+ endpoints), hosted MCP server, and data exports. Free tier: 100 API calls/day. Step 8: Next Steps ------------------ You just learned how to install EdgarTools, look up a company, get financial statements, and browse filings. Here's where to go next: **Financial Data** * [Choosing the Right API](https://edgartools.readthedocs.io/en/latest/xbrl/getting-started/choosing-the-right-api/) — Which method to use for your task (start here!) * [Financial Statements Guide](https://edgartools.readthedocs.io/en/latest/guides/financial-data/) — Income statements, balance sheets, cash flow in depth * [Extract Statements from Filings](https://edgartools.readthedocs.io/en/latest/guides/extract-statements/) — XBRL data extraction * [Company Facts](https://edgartools.readthedocs.io/en/latest/guides/company-facts/) — Historical financial data across all filings **Companies & Filings** * [Find a Company](https://edgartools.readthedocs.io/en/latest/guides/finding-companies/) — Search by name, ticker, CIK, industry, or exchange * [Working with Filings](https://edgartools.readthedocs.io/en/latest/guides/working-with-filing/) — Open, view, and parse any SEC filing * [Search & Filter Filings](https://edgartools.readthedocs.io/en/latest/guides/searching-filings/) — Find exactly the filings you need **Filing Types** * [Annual & Quarterly Reports](https://edgartools.readthedocs.io/en/latest/concepts/data-objects/) — 10-K and 10-Q data objects * [Current Events (8-K)](https://edgartools.readthedocs.io/en/latest/eightk-filings/) — Material events and press releases * [Insider Trades (Form 4)](https://edgartools.readthedocs.io/en/latest/insider-filings/) — Monitor insider transactions * [Institutional Holdings (13F)](https://edgartools.readthedocs.io/en/latest/guides/thirteenf-data-object-guide/) — Who owns what **Reference** * [Cheat Sheet](https://edgartools.readthedocs.io/en/latest/quick-guide/) — Common operations at a glance * [Notebooks](https://edgartools.readthedocs.io/en/latest/notebooks/) — Interactive Colab tutorials you can run in your browser Getting Help ------------ * **[Documentation](https://edgartools.readthedocs.io/en/latest/) **: Browse our comprehensive guides * **[GitHub Discussions](https://github.com/dgunning/edgartools/discussions) **: Ask questions and share insights * **[Issues](https://github.com/dgunning/edgartools/issues) **: Report bugs or request features Support EdgarTools ------------------ If you found this quickstart helpful, consider supporting EdgarTools development: [![Buy Me A Coffee](https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png)](https://www.buymeacoffee.com/edgartools) Back to top --- # Python SEC Filings Tutorials | Free Colab Notebooks | EdgarTools - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/notebooks/#sec-filings-python-tutorials-free-colab-notebooks) SEC Filings Python Tutorials — Free Colab Notebooks =================================================== Access SEC EDGAR data with Python — completely **free**, no API key or paid subscription required. Every notebook runs instantly in **Google Colab** with one click. Install locally with `pip install edgartools`. [**Get Started**](https://edgartools.readthedocs.io/en/stable/notebooks/#getting-started) | [View on GitHub](https://github.com/dgunning/edgartools/tree/main/notebooks) * * * Getting Started --------------- New to edgartools? Start here. These notebooks cover installation, company lookups, and your first SEC filing queries. | Notebook | Difficulty | Links | | --- | --- | --- | | **SEC EDGAR API in Python** — Comprehensive overview: companies, filings, financials, insiders, and holdings in one notebook | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-edgar-api-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/sec-edgar-api-python.ipynb) | | **SEC Company Data with Python** — Look up any company by ticker or CIK: metadata, filing history, industry classification | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-company-data-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/sec-company-data-python.ipynb) | | **Getting Started with SEC Filings** — First steps: install, configure, and pull your first filing | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/01_getting_started.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/01_getting_started.ipynb) | | **Beginner's Guide to EdgarTools** — Complete walkthrough of the core API | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Beginners-Guide.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/Beginners-Guide.ipynb) | | **Search SEC Filings by Ticker Symbol** — Find filings by ticker, CIK, or company name | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Ticker-Search-with-edgartools.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/Ticker-Search-with-edgartools.ipynb) | | **Troubleshooting SSL Issues** — Fix SSL/TLS connection problems in corporate or restricted environments | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/02_troubleshooting_ssl.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/02_troubleshooting_ssl.ipynb) | * * * Filings ------- Search, filter, download, and analyze SEC filings. From today's filings to bulk downloads to full 10-K/10-Q/8-K parsing. | Notebook | Difficulty | Links | | --- | --- | --- | | **Search and Filter SEC Filings** — Search across all companies by date, form type, or quarter | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/search-sec-filings-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/search-sec-filings-python.ipynb) | | **Get Today's SEC Filings** — Real-time access to filings submitted today | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-filings-today-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/sec-filings-today-python.ipynb) | | **Monitor Filings for Multiple Companies** — Watch multiple tickers for new SEC submissions | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/monitor-sec-filings-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/monitor-sec-filings-python.ipynb) | | **Download 10-K Annual Reports** — Download and parse 10-K annual reports | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/download-10k-annual-report-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/download-10k-annual-report-python.ipynb) | | **Analyze 10-K Annual Reports** — Extract business description, risk factors, MD&A, and financials | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/analyze-10k-annual-report-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/analyze-10k-annual-report-python.ipynb) | | **Extract Business Description from 10-K Item 1** — Pull the company overview section from annual reports | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/10k-business-description-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/10k-business-description-python.ipynb) | | **10-Q Quarterly Earnings** — Parse quarterly financial data from 10-Q filings | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/10q-quarterly-earnings-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/10q-quarterly-earnings-python.ipynb) | | **Extract 8-K Earnings Releases** — Pull earnings announcements from 8-K current event reports | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/8k-earnings-release-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/8k-earnings-release-python.ipynb) | | **Download SEC Filings in Bulk** — Batch download filings across companies and date ranges | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/download-sec-filings-bulk-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/download-sec-filings-bulk-python.ipynb) | | **Filter Companies by Industry and SIC Code** — Find companies by sector using SEC industry classifications | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-industry-sic-code-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/sec-industry-sic-code-python.ipynb) | | **SEC Filing Text for NLP** — Extract raw text from filings for natural language processing | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-filing-text-nlp-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/sec-filing-text-nlp-python.ipynb) | | **Filing Exhibits and Attachments** — Access exhibits, press releases, and supplemental documents | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-filing-exhibits-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/sec-filing-exhibits-python.ipynb) | | **Analyze SEC Comment Letters** — Parse CORRESP filings: SEC staff questions and company responses | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-comment-letters-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/sec-comment-letters-python.ipynb) | | **Browse and Page Through Filings** — Navigate large filing collections with paging | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Paging-Through-Filings.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/Paging-Through-Filings.ipynb) | | **Working with Filing Attachments** — Access individual documents within a filing | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Beginners-filings-attachments.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/Beginners-filings-attachments.ipynb) | * * * Financial Statements -------------------- Extract income statements, balance sheets, and cash flow statements from SEC filings. Compare financials across companies and time periods. | Notebook | Difficulty | Links | | --- | --- | --- | | **Financial Statements from SEC Filings** — Extract income statements, balance sheets, and cash flows from 10-K/10-Q | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/financial-statements-sec-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/financial-statements-sec-python.ipynb) | | **Extract Revenue and Earnings** — Pull revenue, net income, and EPS from SEC filings | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/extract-revenue-earnings-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/extract-revenue-earnings-python.ipynb) | | **Compare Company Financials** — Side-by-side financial comparisons across companies | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/compare-company-financials-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/compare-company-financials-python.ipynb) | | **Viewing Financial Statements** — Display and navigate financial statement tables | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Viewing-Financial-Statements.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/Viewing-Financial-Statements.ipynb) | | **Financial Statements to DataFrame** — Build quarterly IS, BS, and CF DataFrames with multi-index (Ticker, Period) for multiple companies | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/financial-statements-to-dataframe.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/financial-statements-to-dataframe.ipynb) | Prefer a visual interface? Every tutorial above runs as Python code. **[edgar.tools](https://app.edgar.tools/?utm_source=edgartools-docs&utm_medium=see-live&utm_content=notebooks) ** puts the same data in a web UI — no code, no notebooks, no setup. * **[Browse any company's filings and financials →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=notebooks) ** * **[See filings arrive in real time →](https://app.edgar.tools/filings?utm_source=edgartools-docs&utm_medium=see-live&utm_content=notebooks) ** * **[Explore insider trades with sentiment analysis →](https://app.edgar.tools/companies/TSLA?utm_source=edgartools-docs&utm_medium=see-live&utm_content=notebooks) ** Free tier available. Also includes a REST API and hosted MCP server for AI integrations. * * * XBRL Deep Dive -------------- Parse structured XBRL financial data — fact queries, multi-period views, custom tags, and advanced statement analysis. | Notebook | Difficulty | Links | | --- | --- | --- | | **Parse XBRL Financial Data** — Work with XBRL-tagged data: income statements, balance sheets, disclosures | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/xbrl-financial-data-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/xbrl-financial-data-python.ipynb) | | **Read Data from XBRL** — Extract structured data from XBRL documents | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Reading-Data-From-XBRL.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/Reading-Data-From-XBRL.ipynb) | | **XBRL Fact Queries** — Query individual XBRL facts with the enhanced API | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/XBRL2-FactQueries.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/XBRL2-FactQueries.ipynb) | | **Multi-Period Financial Views** — Compare financial statements across quarters and years | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/XBRL2-PeriodViews.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/XBRL2-PeriodViews.ipynb) | | **Cash Flow Statements** — Analyze cash flow statements in detail | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/XBRL2-Cashflow-Statements.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/XBRL2-Cashflow-Statements.ipynb) | | **Standardized Financial Statements** — Map company-specific tags to standardized concepts | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/XBRL2-StandardizedStatements.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/XBRL2-StandardizedStatements.ipynb) | | **Quarterly Financial Statements** — Quarterly statement analysis and comparison | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/XBRL2-QuarterlyStatements.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/XBRL2-QuarterlyStatements.ipynb) | | **Stitch Statements Across Filings** — Combine statements from multiple filings into time series | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/XBRL2-StitchingStatements.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/XBRL2-StitchingStatements.ipynb) | | **Custom XBRL Tags** — Handle company-specific extension tags | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/XBRL2-CustomTags.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/XBRL2-CustomTags.ipynb) | | **Non-Financial Statements** — Segment disclosures and non-financial XBRL data | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/XBRL2-NonFinancialStatements.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/XBRL2-NonFinancialStatements.ipynb) | | **Instance-Only XBRL** — Parse XBRL documents without a taxonomy schema | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/XBRL2-Instance-Only-XBRL.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/XBRL2-Instance-Only-XBRL.ipynb) | | **Explore XBRL Concepts** — Browse the XBRL taxonomy and concept hierarchy | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/XBRLConcepts.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/XBRLConcepts.ipynb) | * * * Insider Trading --------------- Track insider buying, selling, and ownership changes from SEC Form 3 and Form 4 filings. | Notebook | Difficulty | Links | | --- | --- | --- | | **Track Insider Trading from Form 4** — Monitor officer and director buys, sells, and option exercises | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/insider-trading-sec-form4-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/insider-trading-sec-form4-python.ipynb) | | **Initial Insider Ownership (Form 3)** — Analyze initial ownership disclosures for new insiders | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Initial-Insider-Transactions.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/Initial-Insider-Transactions.ipynb) | * * * Beneficial Ownership -------------------- Track large shareholders and activist investors through SEC Schedule 13D/G filings. | Notebook | Difficulty | Links | | --- | --- | --- | | **Beneficial Ownership (Schedule 13D/G)** — Track 5%+ shareholders, activist positions, and ownership changes | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/beneficial-ownership-sec-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/beneficial-ownership-sec-python.ipynb) | * * * Institutional Holdings ---------------------- Analyze what hedge funds, mutual funds, and large investors are buying and selling. | Notebook | Difficulty | Links | | --- | --- | --- | | **13F Institutional Holdings** — Quarterly portfolio disclosures from hedge funds and institutional investors | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/13f-institutional-holdings-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/13f-institutional-holdings-python.ipynb) | * * * Investment Funds ---------------- Analyze mutual fund, ETF, and closed-end fund portfolios from SEC N-PORT and other fund filings. | Notebook | Difficulty | Links | | --- | --- | --- | | **Mutual Fund Holdings (N-PORT)** — Complete portfolio holdings from monthly N-PORT filings | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/mutual-fund-holdings-nport-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/mutual-fund-holdings-nport-python.ipynb) | | **Money Market Fund Holdings (N-MFP)** — Portfolio holdings, yields, NAV, and liquidity from monthly N-MFP filings | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/money-market-fund-nmfp-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/money-market-fund-nmfp-python.ipynb) | | **ETF and Fund Holdings** — Analyze ETF portfolios and fund composition | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/etf-fund-holdings-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/etf-fund-holdings-python.ipynb) | | **Fund Census (N-CEN)** — Annual fund census: series, service providers, directors, ETF mechanics, broker commissions | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/fund-census-ncen-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/fund-census-ncen-python.ipynb) | | **Fund Filing Types** — Overview of SEC fund filing types and structures | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Fund-Filings.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/Fund-Filings.ipynb) | | **Fund Derivative Holdings** — Analyze derivative positions within fund portfolios | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Fund-Derivatives.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/Fund-Derivatives.ipynb) | * * * Business Development Companies ------------------------------ Analyze BDC portfolio investments, lending activity, and SEC filings. | Notebook | Difficulty | Links | | --- | --- | --- | | **BDC SEC Filings and Portfolios** — Analyze BDC investment portfolios and lending activity | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/bdc-business-development-company-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/bdc-business-development-company-python.ipynb) | * * * Executive Compensation & Proxy Statements ----------------------------------------- Parse proxy statements for executive pay, board composition, and shareholder proposals. | Notebook | Difficulty | Links | | --- | --- | --- | | **Proxy Statements (DEF 14A)** — Parse proxy statements: proposals, board members, and voting items | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/proxy-statement-def14a-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/proxy-statement-def14a-python.ipynb) | | **Executive Compensation** — Extract CEO and executive pay from proxy statements | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/executive-compensation-sec-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/executive-compensation-sec-python.ipynb) | * * * Why EdgarTools? --------------- **sec-api.io** is a data access API -- it gives you JSON from REST endpoints, and you build the analysis yourself. **EdgarTools** is a data analysis library -- it parses filings into structured Python objects with built-in methods for the analysis you actually want to do. And it's free. ### The Core Difference With sec-api, getting 13F institutional holdings means calling an endpoint, receiving JSON, then writing code to compare quarters, calculate position changes, and format results. With edgartools, that analysis is built in: `from edgar import * # Parse a 13F filing into a structured object thirteenf = Company("BERKSHIRE HATHAWAY").get_filings(form="13F-HR")[0].obj() # Built-in quarter-over-quarter comparison thirteenf.compare_holdings() # NEW, CLOSED, INCREASED, DECREASED positions # Multi-quarter trend analysis with sparklines thirteenf.holding_history(periods=4) # All holdings as a pandas DataFrame, ready for analysis thirteenf.holdings_data()` sec-api returns the raw holdings data as JSON. The comparison logic, trend analysis, and DataFrame conversion are left to you. ### What You Get Out of the Box EdgarTools doesn't just fetch data -- it structures it into objects with properties, methods, and DataFrames designed for the analysis Python developers actually do: | Filing Type | What edgartools gives you | What a JSON API gives you | | --- | --- | --- | | **10-K / 10-Q** | `TenK` / `TenQ` objects with `.financials`, section extraction, multi-period statements | Raw XBRL JSON -- you build the statement structure | | **8-K** | `EightK` with item-level parsing, earnings extraction | Section text or structured fields for a few items | | **13F** | `ThirteenF` with `compare_holdings()`, `holding_history()`, sparklines | Holdings array -- you write the diff logic | | **N-PORT** | `FundReport` with `investment_data()`, asset allocation, country exposure | Holdings array -- you aggregate and categorize | | **N-MFP** | `MoneyMarketFund` with yield/NAV/liquidity time series, category breakdowns | \-- | | **N-CEN** | `FundCensus` with series, providers, broker commissions, board composition | \-- | | **DEF 14A** | `ProxyStatement` with executive compensation tables, board data | Separate exec comp and board endpoints | | **13D/G** | `Schedule13DG` with ownership parsing | Structured JSON | | **Form 4** | `Ownership` with transaction details | Structured JSON | ### Pricing | | EdgarTools | sec-api.io | | --- | --- | --- | | **Price** | Free forever | Free trial (100 calls), then $49-$239/mo | | **API key** | Not required | Required | | **Open source** | Yes (MIT license) | No | | **Works offline** | Yes (with local storage) | No | ### Use sec-api if you need... sec-api is a hosted platform backed by databases and infrastructure, so it can offer things a client-side library can't: real-time WebSocket filing streams, full-text boolean search across all filings, filing-to-PDF conversion, and prebuilt datasets like SEC enforcement actions and Form ADV investment adviser data. It also works with any programming language via REST, not just Python. ### Use EdgarTools if you need... EdgarTools is for Python developers who want to go straight from a filing to analysis. No API key, no HTTP plumbing, no JSON wrangling -- just `pip install edgartools` and you get structured objects with built-in analysis methods, pandas DataFrames, Rich terminal display, and native Jupyter/Colab support. Free forever, open source, and works offline. * * * Running on Google Colab ----------------------- Click any **Open in Colab** badge above, or: 1. Go to [colab.research.google.com](https://colab.research.google.com/) 2. **File > Open notebook > GitHub** tab 3. Enter `dgunning/edgartools` and select a notebook Resources --------- * [EdgarTools Documentation](https://edgartools.io/) * [GitHub Repository](https://github.com/dgunning/edgartools) * [PyPI Package](https://pypi.org/project/edgartools/) * [Installation Guide](https://edgartools.readthedocs.io/en/stable/installation/) * [Quick Start](https://edgartools.readthedocs.io/en/stable/quickstart/) Back to top --- # Overview - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/financial-data/#financial-statements-income-balance-sheet-and-cash-flow-from-sec-filings) Financial Statements: Income, Balance Sheet, and Cash Flow from SEC Filings =========================================================================== **Get financial statements from SEC filings in Python** Quick Start ----------- Three lines to Apple's income statement: `from edgar import Company company = Company("AAPL") financials = company.get_financials() income_statement = financials.income_statement()` ![AAPL Income Statement](https://edgartools.readthedocs.io/en/stable/images/aapl-income-xbrl.webp) That's it. You now have Apple's full income statement from their latest 10-K filing. > **[See Apple's financials on edgar.tools — no code required →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=financial-data) > ** * * * Get Specific Values ------------------- Need just one number? Use the convenience methods: `financials = company.get_financials() revenue = financials.get_revenue() net_income = financials.get_net_income() total_assets = financials.get_total_assets()` | Method | Returns | | --- | --- | | `get_revenue()` | Total revenue / net sales | | `get_net_income()` | Net income | | `get_operating_income()` | Operating income / loss | | `get_total_assets()` | Total assets | | `get_total_liabilities()` | Total liabilities | | `get_stockholders_equity()` | Stockholders' equity | | `get_operating_cash_flow()` | Operating cash flow | | `get_free_cash_flow()` | Free cash flow | | `get_capital_expenditures()` | Capital expenditures | | `get_current_assets()` | Current assets | | `get_current_liabilities()` | Current liabilities | All methods accept `period_offset` to access prior periods (0=current, 1=previous). * * * Available Statements -------------------- `financials = company.get_financials() income = financials.income_statement() balance = financials.balance_sheet() cashflow = financials.cashflow_statement() equity = financials.statement_of_equity() comprehensive = financials.comprehensive_income()` | Statement | Method | | --- | --- | | Income Statement | `income_statement()` | | Balance Sheet | `balance_sheet()` | | Cash Flow Statement | `cashflow_statement()` | | Statement of Equity | `statement_of_equity()` | | Comprehensive Income | `comprehensive_income()` | Cash flow naming The method is `cashflow_statement()` (no underscore between "cash" and "flow"). The `Company` object also accepts `company.cash_flow()` as an alias, but on the `Financials` object always use `cashflow_statement()`. * * * Export to DataFrame ------------------- Every statement converts to a pandas DataFrame: `income = financials.income_statement() # Convert to DataFrame df = income.to_dataframe() # Export to CSV df.to_csv("apple_income_statement.csv") # Export to Excel df.to_excel("apple_income_statement.xlsx")` The DataFrame preserves the statement structure with labeled rows and period columns. * * * Quarterly Financials -------------------- Use `get_quarterly_financials()` for 10-Q data: `quarterly = company.get_quarterly_financials() income = quarterly.income_statement()` ![AAPL Quarterly Income Statement](https://edgartools.readthedocs.io/en/stable/images/aapl-income-quarterly-xbrl.webp) | Need | Method | | --- | --- | | Annual (10-K) | `company.get_financials()` | | Quarterly (10-Q) | `company.get_quarterly_financials()` | * * * Getting Different Levels of Detail ---------------------------------- Financial statements can show different levels of detail. Use the `view` parameter to control what you see: `financials = company.get_financials() # Summary: Matches SEC Viewer (~15-20 rows) income = financials.income_statement(view="summary") # Standard: Matches the filing document (default) income = financials.income_statement(view="standard") # Detailed: All dimensional breakdowns income = financials.income_statement(view="detailed")` ### Summary View ![AAPL Detailed Income Statement](https://edgartools.readthedocs.io/en/stable/images/aapl-income-summary.webp) ### Detailed View ![AAPL Detailed Income Statement](https://edgartools.readthedocs.io/en/stable/images/aapl-income-detailed.webp) | View | Shows | Typical Rows | Best For | | --- | --- | --- | --- | | `"summary"` | Matches SEC Viewer | ~15-20 | Quick overview, validation | | `"standard"` | Matches filing document | ~25-35 | Display, full context | | `"detailed"` | All dimensional breakdowns | ~50+ | Data extraction, segment analysis | ### Example: Apple Revenue With `view="summary"`: `Revenue $391,035M` With `view="standard"`: `Revenue: Products $298,085M Services $92,950M` With `view="detailed"`: `Revenue: Products $298,085M iPhone $201,183M Mac $29,357M iPad $26,694M Wearables, Home and Accessories $40,851M Services $92,950M` ### Views with DataFrames The `view` parameter also works when exporting to DataFrame: `income = financials.income_statement() df_summary = income.to_dataframe(view="summary") df_standard = income.to_dataframe(view="standard") df_detailed = income.to_dataframe(view="detailed")` ### When to Use Each View * **Summary**: Quick checks, matches SEC Viewer, comparing many companies * **Standard**: Matches the filing document, full context with face-level dimensions * **Detailed**: Data extraction, segment analysis, complete dimensional breakdowns **Learn more:** [Dimension Handling Guide](https://edgartools.readthedocs.io/en/stable/xbrl/concepts/dimension-handling/) * * * Compare Multiple Periods ------------------------ To analyze trends across multiple filings, use `XBRLS`: `from edgar.xbrl import XBRLS # Get last 3 annual filings filings = company.get_filings(form="10-K").head(3) # Stitch them together xbrls = XBRLS.from_filings(filings) # Get income statement across all periods income = xbrls.statements.income_statement() # Use view="detailed" for dimensional breakdowns across periods income_detailed = xbrls.statements.income_statement(view="detailed")` The `view` parameter works the same as on single-period statements (`"standard"`, `"detailed"`, `"summary"`). This aligns the periods and concepts across filings for easy comparison. **Learn more:** [Multi-Period Analysis Guide](https://edgartools.readthedocs.io/en/stable/xbrl/guides/multi-period-analysis/) * * * ### Why skip amendments? Use `amendments=False` when fetching filings. Amended filings (10-K/A) sometimes contain only the corrected sections, not complete financial statements. * * * Advanced Topics --------------- ### Raw Facts Query Query individual XBRL facts for research or custom calculations: `xbrl = filing.xbrl() # Find all revenue facts revenue_facts = xbrl.facts.query()\ .by_concept("Revenue")\ .to_dataframe() # Search by label rd_facts = xbrl.facts.query()\ .by_label("Research", exact=False)\ .to_dataframe()` **Learn more:** [XBRL Querying Guide](https://edgartools.readthedocs.io/en/stable/xbrl-querying/) * * * Troubleshooting --------------- ### "No financial data found" Some companies (especially newer or smaller ones) may not have XBRL data: `filing = company.get_filings(form="10-K").latest() if filing.xbrl(): print("XBRL available") else: print("No XBRL - try filing.text() for raw content")` ### "Statement is empty" Try using the detailed view to include all dimensional data: `df = income.to_dataframe(view="detailed")` ### "Numbers don't match the SEC website" Check that you're looking at the right period: `xbrl = filing.xbrl() print(xbrl.reporting_periods)` * * * API Quick Reference ------------------- ### Company-Level (Easiest) | Method | Description | | --- | --- | | `company.get_financials()` | Latest annual financials (10-K) | | `company.get_quarterly_financials()` | Latest quarterly financials (10-Q) | ### Financials Object | Method | Description | | --- | --- | | `financials.income_statement()` | Income statement | | `financials.balance_sheet()` | Balance sheet | | `financials.cashflow_statement()` | Cash flow statement | | `financials.get_revenue()` | Revenue value | | `financials.get_net_income()` | Net income value | | `financials.get_total_assets()` | Total assets value | | `financials.get_financial_metrics()` | Dict of all key metrics | ### Statement Object | Method | Description | | --- | --- | | `statement.to_dataframe()` | Convert to pandas DataFrame | | `statement.to_dataframe(view="summary")` | Matches SEC Viewer | | `statement.to_dataframe(view="standard")` | Matches filing document | | `statement.to_dataframe(view="detailed")` | All dimensional breakdowns | ### Filing-Level (More Control) | Method | Description | | --- | --- | | `filing.xbrl()` | Parse XBRL from filing | | `xbrl.statements.income_statement()` | Get income statement | | `xbrl.facts.query()` | Query individual facts | ### Multi-Period Analysis | Method | Description | | --- | --- | | `XBRLS.from_filings(filings)` | Stitch multiple filings together | | `xbrls.statements.income_statement()` | Aligned multi-period statement | * * * Next Steps ---------- * **[Stock Splits & EPS Normalization](https://edgartools.readthedocs.io/en/stable/guides/stock-splits-eps-normalization/) ** – Detect splits and normalize per-share metrics * **[Multi-Period Analysis](https://edgartools.readthedocs.io/en/stable/xbrl/guides/multi-period-analysis/) ** – Build custom time series * **[Standardization](https://edgartools.readthedocs.io/en/stable/xbrl/concepts/standardization/) ** – How cross-company comparison works * **[XBRL Documentation](https://edgartools.readthedocs.io/en/stable/xbrl/) ** – Complete XBRL reference * * * See it on edgar.tools The code above extracts financials programmatically. **edgar.tools** shows the same data visually — plus Disclosure Search, which lets you browse XBRL topics across every 10-K year for a company. * **[See Apple's financials →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=financial-data) ** * **[Browse income tax disclosures across all filing years →](https://app.edgar.tools/disclosures/income-taxes?utm_source=edgartools-docs&utm_medium=see-live&utm_content=financial-data) ** * **[Explore all 12 XBRL disclosure topics →](https://app.edgar.tools/disclosures?utm_source=edgartools-docs&utm_medium=see-live&utm_content=financial-data) ** Export to Excel, PDF, or CSV with one click. Free tier available. [Pricing →](https://app.edgar.tools/pricing?utm_source=edgartools-docs&utm_medium=see-live&utm_content=financial-data) Back to top --- # MCP Server & Skills - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/ai-integration/#ai-integration) AI Integration ============== EdgarTools provides two AI integration features: 1. **MCP Server** -- Gives any MCP-compatible AI client direct access to SEC filing data through specialized tools 2. **Skills** -- Teaches Claude how to write better EdgarTools code by providing structured patterns and best practices Both are optional. Install with: `pip install "edgartools[ai]"` MCP Server ---------- The [Model Context Protocol](https://modelcontextprotocol.io/) server gives any MCP-compatible AI client access to SEC filing data -- whether that's a developer's Claude Desktop, a team's shared server, or a containerized deployment. No code required. ### Setup Choose the deployment method that fits your use case: #### uvx (recommended -- zero install) Ideal for individual use or scripted deployment. Requires [uv](https://docs.astral.sh/uv/getting-started/installation/) . `{ "mcpServers": { "edgartools": { "command": "uvx", "args": ["--from", "edgartools[ai]", "edgartools-mcp"], "env": { "EDGAR_IDENTITY": "Your Name your.email@example.com" } } } }` If you get a "spawn uvx ENOENT" error on macOS, use the full path to uvx (find it with `which uvx`). #### Python When edgartools is already installed in your environment: `{ "mcpServers": { "edgartools": { "command": "python3", "args": ["-m", "edgar.ai"], "env": { "EDGAR_IDENTITY": "Your Name your.email@example.com" } } } }` On Windows, use `python` instead of `python3`. #### Docker For server or production deployments where you want isolation and reproducibility: `FROM python:3.12-slim RUN pip install "edgartools[ai]" ENV EDGAR_IDENTITY="Your Name your.email@example.com" ENTRYPOINT ["python", "-m", "edgar.ai"]` Build and run: `docker build -t edgartools-mcp . docker run -i edgartools-mcp` The community also maintains Docker images -- see [hackerdogs/edgartools-mcp](https://hub.docker.com/r/hackerdogs/edgartools-mcp) on Docker Hub for a ready-to-use container with config templates for multiple MCP clients. #### HTTP transport (remote / team deployment) For shared servers, remote deployment, or registry-listed instances, run with Streamable HTTP transport instead of stdio: `edgartools-mcp --transport streamable-http --port 8000` This starts the server on `http://0.0.0.0:8000/mcp`. Clients connect with a URL instead of launching a subprocess: `{ "mcpServers": { "edgartools": { "url": "http://your-server:8000/mcp" } } }` CLI flags: | Flag | Default | Description | | --- | --- | --- | | `--transport` | `stdio` | `stdio` or `streamable-http` | | `--host` | `0.0.0.0` | Bind address for HTTP server | | `--port` | `8000` | Listen port for HTTP server | Docker with HTTP transport: `FROM python:3.12-slim RUN pip install "edgartools[ai]" ENV EDGAR_IDENTITY="Your Name your.email@example.com" ENTRYPOINT ["edgartools-mcp", "--transport", "streamable-http"] EXPOSE 8000` Replace `Your Name your.email@example.com` with your actual name and email. The SEC requires this to identify API users. **Verify** `python -m edgar.ai --test` **For Claude Desktop**, add the config above to your config file (`~/Library/Application Support/Claude/claude_desktop_config.json` on macOS, `%APPDATA%\Claude\claude_desktop_config.json` on Windows) and restart. You should see the MCP tools icon in the chat input. > **[edgar.tools also runs a hosted MCP server with AI-enriched data — no local setup needed →](https://app.edgar.tools/docs/mcp/setup?utm_source=edgartools-docs&utm_medium=see-live&utm_content=ai-integration) > ** ### Available Tools #### edgar\_company Get company profile, financials, recent filings, and ownership in one call. | Parameter | Description | | --- | --- | | `identifier` | Ticker, CIK, or company name (required) | | `include` | Sections to return: `profile`, `financials`, `filings`, `ownership` | | `periods` | Number of financial periods (default: 4) | | `annual` | Annual vs quarterly data (default: true) | **Try asking Claude:** * "Show me Apple's profile and latest financials" * "Get Microsoft's recent filings and ownership data" #### edgar\_search Search for companies or filings. | Parameter | Description | | --- | --- | | `query` | Search keywords (required) | | `search_type` | `companies`, `filings`, or `all` | | `identifier` | Limit to a specific company | | `form` | Filter by form type (e.g., `10-K`, `8-K`) | | `limit` | Max results (default: 10) | **Try asking Claude:** * "Search for semiconductor companies" * "Find Apple's 10-K filings" #### edgar\_filing Read filing content or specific sections. | Parameter | Description | | --- | --- | | `accession_number` | SEC accession number | | `identifier` + `form` | Alternative: company + form type | | `sections` | `summary`, `business`, `risk_factors`, `mda`, `financials`, or `all` | **Try asking Claude:** * "Show me the risk factors from Apple's latest 10-K" * "Get the MD&A section from Tesla's most recent annual report" #### edgar\_compare Compare companies side-by-side or analyze an industry. | Parameter | Description | | --- | --- | | `identifiers` | List of tickers/CIKs to compare | | `industry` | Alternative: industry name | | `metrics` | Metrics to compare (e.g., `revenue`, `net_income`) | | `periods` | Number of periods (default: 4) | **Try asking Claude:** * "Compare Apple, Microsoft, and Google on revenue and net income" * "How do the top semiconductor companies compare?" #### edgar\_ownership Insider transactions or fund portfolios. | Parameter | Description | | --- | --- | | `identifier` | Ticker, CIK, or fund CIK (required) | | `analysis_type` | `insiders`, `fund_portfolio`, or `portfolio_diff` | | `limit` | Max results (default: 20) | **Try asking Claude:** * "Show me recent insider transactions at Apple" * "What stocks does Berkshire Hathaway hold?" #### edgar\_monitor Get the latest SEC filings in real-time. | Parameter | Description | | --- | --- | | `form` | Filter by form type (e.g., `8-K`, `4`) | | `limit` | Max results (default: 20) | **Try asking Claude:** * "What SEC filings were just submitted?" * "Show me recent 8-K filings" #### edgar\_trends Get financial time series with growth rates. | Parameter | Description | | --- | --- | | `identifier` | Ticker, CIK, or company name (required) | | `concepts` | Metrics to track (e.g., `revenue`, `net_income`, `eps`) | | `periods` | Number of periods (default: 5) | **Try asking Claude:** * "Show me Apple's revenue trend over 5 years" * "What is Microsoft's EPS growth trajectory?" #### edgar\_screen Discover companies by industry, exchange, or state. | Parameter | Description | | --- | --- | | `industry` | Industry keyword | | `exchange` | Exchange name (e.g., `NYSE`, `Nasdaq`) | | `state` | State of incorporation (2-letter code) | | `limit` | Max results (default: 20) | **Try asking Claude:** * "Find pharmaceutical companies on NYSE" * "What software companies are in Delaware?" #### edgar\_text\_search Full-text search across SEC filing content. | Parameter | Description | | --- | --- | | `query` | Search text (required) | | `identifier` | Limit to a specific company | | `forms` | Filter by form types (e.g., `["8-K", "10-K"]`) | | `start_date` | Start date filter | **Try asking Claude:** * "Search for filings mentioning artificial intelligence" * "Find 8-K filings about cybersecurity incidents" #### edgar\_fund Get fund, ETF, BDC, and money market fund data. | Parameter | Description | | --- | --- | | `action` | `lookup`, `search`, `portfolio`, `money_market`, `bdc_search`, or `bdc_portfolio` (required) | | `identifier` | Fund ticker, series ID, or CIK | | `query` | Search text for fund or BDC name | | `limit` | Max results (default: 20) | **Try asking Claude:** * "Look up the Vanguard 500 Index Fund" * "Show me SPY's portfolio holdings" #### edgar\_proxy Get executive compensation and governance data from DEF 14A proxy statements. | Parameter | Description | | --- | --- | | `identifier` | Ticker, CIK, or company name (required) | | `filing_index` | Which proxy filing, 0=latest (default: 0) | **Try asking Claude:** * "What is Apple's CEO compensation?" * "Show me Microsoft's pay vs performance data" ### Any MCP Client The server works with any MCP-compatible client -- Claude Desktop, Cline, Continue.dev, or your own tooling. The configuration is the same `mcpServers` block regardless of client: `{ "mcpServers": { "edgartools": { "command": "uvx", "args": ["--from", "edgartools[ai]", "edgartools-mcp"], "env": { "EDGAR_IDENTITY": "Your Name your.email@example.com" } } } }` Where it goes depends on the client: Claude Desktop config file, Cline MCP settings, `~/.continue/config.json`, etc. edgar.tools also runs a hosted MCP server The local edgartools MCP server queries EDGAR directly through Python. The **[edgar.tools hosted MCP server](https://app.edgar.tools/docs/mcp/setup?utm_source=edgartools-docs&utm_medium=see-live&utm_content=ai-integration) ** adds AI-enriched data processed server-side: | Capability | Local (edgartools) | Hosted (edgar.tools) | | --- | --- | --- | | Material events | Basic 8-K parsing | LLM-classified event types | | Disclosure search | — | 12 XBRL topic clusters, all years | | Insider data | Individual Form 4s | 802K+ transactions with sentiment | | Filing sections | Raw text | AI summaries and key takeaways | Free tier: truncated MCP responses. Professional ($24.99/mo): full results. **[Set up the hosted MCP server →](https://app.edgar.tools/docs/mcp/setup?utm_source=edgartools-docs&utm_medium=see-live&utm_content=ai-integration) ** Skills ------ Skills are structured documentation packages that teach Claude how to write better EdgarTools code. They guide Claude to use the right APIs, avoid common mistakes, and follow best practices. ### What Do Skills Do? Without skills, Claude might write verbose code using low-level APIs: `# Without skills -- verbose, fragile facts = company.get_facts() income = facts.income_statement(periods=1, annual=True) if income is not None and not income.empty: if 'Revenue' in income.columns: revenue = income['Revenue'].iloc[0]` With skills, Claude writes idiomatic code: `# With skills -- clean, correct financials = company.get_financials() revenue = financials.get_revenue()` Skills cover patterns, sharp edges (common mistakes), and API routing decisions across six domains. ### Installing Skills **For Claude Code** (auto-discovered): `from edgar.ai import install_skill install_skill() # Installs to ~/.claude/skills/edgartools/` **For Claude Desktop** (upload as project knowledge): `from edgar.ai import package_skill package_skill() # Creates edgartools.zip` Upload the ZIP to a Claude Desktop Project. ### Skill Domains | Domain | What It Covers | | --- | --- | | **core** | Company lookup, filing search, API routing, quick reference | | **financials** | Financial statements, metrics, multi-company comparison | | **holdings** | 13F filings, institutional portfolios | | **ownership** | Insider transactions (Form 4), ownership summaries | | **reports** | 10-K, 10-Q, 8-K document sections | | **xbrl** | XBRL fact extraction, statement rendering | ### When to Use Which | I want to... | Use | | --- | --- | | Ask Claude questions about companies/filings | MCP Server | | Have Claude write EdgarTools code for me | Skills | | Both | Install both -- they complement each other | Built-in AI Features -------------------- These work without the `[ai]` extra. ### .docs Property Every major EdgarTools object has a `.docs` property with searchable API documentation: `from edgar import Company company = Company("AAPL") company.docs # Full API reference company.docs.search("financials") # Search for specific topics` Available on: `Company`, `Filing`, `Filings`, `XBRL`, `Statement` ### .to\_context() Method Token-efficient output optimized for LLM context windows: `company = Company("AAPL") # Control detail level company.to_context(detail='minimal') # ~100 tokens company.to_context(detail='standard') # ~300 tokens (default) company.to_context(detail='full') # ~500 tokens # Hard token limit company.to_context(max_tokens=200)` Available on: `Company`, `Filing`, `Filings`, `XBRL`, `Statement`, and most data objects. Troubleshooting --------------- **"EDGAR\_IDENTITY environment variable is required"** Add your name and email to the `env` section of your MCP config. The SEC requires identification for API access. **"Module edgar.ai not found"** Install with AI extras: `pip install "edgartools[ai]"` **"python3: command not found" (Windows)** Use `python` instead of `python3` in your MCP config. **MCP server not appearing in Claude Desktop** 1. Check the config file location is correct for your OS 2. Validate JSON syntax 3. Restart Claude Desktop completely (quit and relaunch) 4. Run `python -m edgar.ai --test` to verify **Skills not being picked up** 1. Verify installation: `ls ~/.claude/skills/edgartools/` 2. For Claude Desktop, upload as ZIP to a Project instead 3. Skills only affect code generation, not conversational responses Back to top --- # Installation - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/installation/#installation) Installation ============ Get started with edgartools in minutes. This guide covers all installation methods and system requirements. System Requirements ------------------- * **Python**: 3.8 or higher Quick Installation ------------------ ### Using pip (Recommended) `pip install edgartools` For the latest features and bug fixes: `pip install -U edgartools` ### Using uv (Fast Alternative) `uv pip install edgartools` Development Installation ------------------------ If you want to contribute or use the latest development version: `# Clone the repository git clone https://github.com/dgunning/edgartools.git cd edgartools # Install in development mode pip install -e . # Or with development dependencies pip install -e ".[dev]"` Verify Installation ------------------- Test your installation by running this simple command: `from edgar import get_filings print("EdgarTools installed successfully!")` Expected output: `EdgarTools installed successfully!` If you see this message, your installation is successful. If you see `ImportError: cannot import name 'get_filings' from 'edgar'` then you have likely installed another package named **edgar** not **edgartools**. If you encounter this error, uninstall the conflicting package and reinstall edgartools: `pip uninstall edgar pip install edgartools` Setting Your Identity --------------------- Before using edgartools, you must set your identity to comply with SEC requirements: ### Method 1: In Python Code `from edgar import set_identity # Use your name and email set_identity("John Doe john.doe@company.com") # Or just your email set_identity("john.doe@company.com")` ### Method 2: Environment Variable Set the `EDGAR_IDENTITY` environment variable: **Linux/macOS:** `export EDGAR_IDENTITY="John Doe john.doe@company.com"` **Windows:** `set EDGAR_IDENTITY=John Doe john.doe@company.com` **Windows PowerShell:** `$env:EDGAR_IDENTITY = "John Doe john.doe@company.com"` Optional Dependencies --------------------- For enhanced functionality, install these optional packages: Troubleshooting --------------- ### Common Issues #### ImportError: No module named 'edgar' **Problem**: Package not installed correctly **Solution**: `pip uninstall edgar pip install --force-reinstall edgartools` #### SEC Identity Error **Problem**: Identity not set **Solution**: Follow the [Setting Your Identity](https://edgartools.readthedocs.io/en/stable/installation/#setting-your-identity) section above #### Permission Errors on Windows **Problem**: Insufficient permissions **Solution**: Run as administrator or use `--user` flag: `pip install --user edgartools` #### SSL Certificate Errors **Problem**: Corporate firewall or proxy **Solution**: Configure pip for your proxy: `pip install --trusted-host pypi.org --trusted-host pypi.python.org edgartools` #### Memory Issues with Large Datasets **Problem**: Out of memory errors **Solution**: - Increase system memory - Use data chunking techniques - Process data in smaller batches ### Getting Help If you encounter issues: 1. **Search existing issues**: [GitHub Issues](https://github.com/dgunning/edgartools/issues) 2. **Create a new issue**: Include Python version, OS, and error messages 3. **Join the community**: Discussions and support channels Virtual Environment Setup ------------------------- For isolated development, use virtual environments: ### Using venv (Python 3.8+) `# Create virtual environment python -m venv edgar-env # Activate (Linux/macOS) source edgar-env/bin/activate # Activate (Windows) edgar-env\Scripts\activate # Install edgartools pip install edgartools # Deactivate when done deactivate` Performance Optimization ------------------------ For optimal performance: 1. **Use Local Storage** to download and work with SEC filings locally 2. **Set reasonable limits** when querying large datasets 3. **Use filtering** to reduce data transfer Next Steps ---------- After installation: 1. **Read the [Quick Start Guide](https://edgartools.readthedocs.io/en/stable/quickstart/) ** for your first analysis 2. **Check the [API Reference](https://edgartools.readthedocs.io/en/stable/api/company/) ** for detailed documentation Security Considerations ----------------------- * **Never commit your identity** to version control * **Use environment variables** for production deployments * **Follow SEC rate limits** to avoid being blocked * **Keep your installation updated** for security patches License ------- EdgarTools is released under the MIT License. See [LICENSE](https://github.com/dgunning/edgartools/blob/main/LICENSE) for details. Back to top --- # Company - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/api/company/#company-api-reference) Company API Reference ===================== The `Company` class is the primary interface for working with public companies in EdgarTools. It provides access to company information, SEC filings, and financial data. Class Overview -------------- `from edgar import Company class Company(Entity): """Represents a public company with SEC filings."""` **Inheritance:** `SecFiler` → `Entity` → `Company` Constructor ----------- ### Company(cik\_or\_ticker) Create a Company instance using either a CIK number or ticker symbol. `Company(cik_or_ticker: Union[str, int])` **Parameters:** - `cik_or_ticker` (Union\[str, int\]): Company identifier - **CIK**: Central Index Key as integer or string (with or without padding) - **Ticker**: Stock ticker symbol (case-insensitive) **Examples:** `# By ticker symbol (case-insensitive) company = Company("AAPL") company = Company("aapl") # By CIK number company = Company(320193) company = Company("320193") company = Company("0000320193") # Zero-padded` **Raises:** - `CompanyNotFoundError`: When company cannot be found - `ValueError`: When identifier format is invalid Core Properties --------------- ### Basic Information #### name `@property def name(self) -> str` Official company name as registered with the SEC. `company = Company("AAPL") print(company.name) # "Apple Inc."` #### cik `@property def cik(self) -> int` Central Index Key - unique identifier assigned by the SEC. `print(company.cik) # 320193` #### display\_name `@property def display_name(self) -> str` Formatted display name combining ticker and company name. `print(company.display_name) # "AAPL - Apple Inc."` #### tickers `@property def tickers(self) -> List[str]` List of all ticker symbols associated with the company. `berkshire = Company("BRK-A") print(berkshire.tickers) # ["BRK-A", "BRK-B"]` ### Industry & Classification #### industry `@property def industry(self) -> str` Industry description based on SIC code. `print(company.industry) # "ELECTRONIC COMPUTERS"` #### sic `@property def sic(self) -> str` Standard Industrial Classification code. `print(company.sic) # "3571"` #### fiscal\_year\_end `@property def fiscal_year_end(self) -> str` Fiscal year end date in MMDD format. `print(company.fiscal_year_end) # "0930" (September 30)` ### Company Status #### is\_company `@property def is_company(self) -> bool` Always `True` for Company instances. Used to distinguish from other entities. `print(company.is_company) # True` #### not\_found `@property def not_found(self) -> bool` Whether the company data was found in SEC database. `print(company.not_found) # False if found, True if not` ### Key Metrics #### shares\_outstanding `@property def shares_outstanding(self) -> Optional[float]` Number of common shares outstanding, sourced from SEC company facts. `company = Company("AAPL") print(company.shares_outstanding) # 15115785000.0` #### public\_float `@property def public_float(self) -> Optional[float]` Public float value in dollars, sourced from SEC company facts. `company = Company("AAPL") print(company.public_float) # 2899948348000.0` Filing Access ------------- ### get\_filings() Get company filings with extensive filtering options. `def get_filings( self, *, year: Union[int, List[int], range] = None, quarter: Union[int, List[int]] = None, form: Union[str, List[str]] = None, accession_number: Union[str, List[str]] = None, file_number: Union[str, List[str]] = None, filing_date: str = None, date: str = None, amendments: bool = True, is_xbrl: bool = None, is_inline_xbrl: bool = None, sort_by: str = "filing_date", trigger_full_load: bool = False ) -> EntityFilings` **Parameters:** - `year`: Filter by year(s) - int, list of ints, or range - `quarter`: Filter by quarter(s) - 1, 2, 3, or 4 - `form`: SEC form type(s) - e.g., "10-K", \["10-K", "10-Q"\] - `accession_number`: Specific accession number(s) - `file_number`: SEC file number(s) - `filing_date`: Date or date range (YYYY-MM-DD or YYYY-MM-DD:YYYY-MM-DD) - `date`: Alias for filing\_date - `amendments`: Include amended filings (default: True) - `is_xbrl`: Filter for XBRL filings - `is_inline_xbrl`: Filter for inline XBRL filings - `sort_by`: Sort field (default: "filing\_date") - `trigger_full_load`: Load all filing details upfront **Returns:** `EntityFilings` - Collection of company filings **Examples:** `# Get all filings all_filings = company.get_filings() # Get specific form types annual_reports = company.get_filings(form="10-K") quarterly_reports = company.get_filings(form=["10-K", "10-Q"]) # Filter by date recent = company.get_filings(filing_date="2023-01-01:") date_range = company.get_filings(filing_date="2023-01-01:2023-12-31") # Filter by year and quarter q4_2023 = company.get_filings(year=2023, quarter=4) multi_year = company.get_filings(year=[2022, 2023]) # XBRL filings only xbrl_filings = company.get_filings(is_xbrl=True) # Exclude amendments original_only = company.get_filings(amendments=False)` ### latest() Get the latest filing(s) of a specific form type. `def latest(self, form: str, n: int = 1) -> Union[Filing, List[Filing]]` **Parameters:** - `form`: SEC form type (e.g., "10-K", "10-Q", "8-K") - `n`: Number of latest filings to return (default: 1) **Returns:** - Single `Filing` if n=1 - `List[Filing]` if n>1 **Examples:** `# Get latest 10-K latest_10k = company.latest("10-K") # Get latest 3 quarterly reports latest_10qs = company.latest("10-Q", 3)` ### Convenience Properties #### latest\_tenk `@property def latest_tenk(self) -> Optional[TenK]` Latest 10-K filing as a TenK object with enhanced functionality. `tenk = company.latest_tenk if tenk: print(tenk.filing_date) financials = tenk.financials` #### latest\_tenq `@property def latest_tenq(self) -> Optional[TenQ]` Latest 10-Q filing as a TenQ object with enhanced functionality. `tenq = company.latest_tenq if tenq: print(tenq.filing_date) financials = tenq.financials` Financial Data -------------- ### get\_financials() Get financial statements from the latest 10-K filing. `def get_financials(self) -> Optional[Financials]` **Returns:** `Financials` object with balance sheet, income statement, and cash flow data **Example:** `financials = company.get_financials() if financials: balance_sheet = financials.balance_sheet income_statement = financials.income cash_flow = financials.cash_flow # Access specific metrics revenue = income_statement.loc['Revenue'].iloc[0] total_assets = balance_sheet.loc['Total Assets'].iloc[0]` ### get\_quarterly\_financials() Get financial statements from the latest 10-Q filing. `def get_quarterly_financials(self) -> Optional[Financials]` **Returns:** `Financials` object from latest quarterly report **Example:** `quarterly = company.get_quarterly_financials() if quarterly: q_income = quarterly.income quarterly_revenue = q_income.loc['Revenue'].iloc[0]` ### get\_facts() Get structured XBRL facts for the company. `def get_facts(self) -> Optional[EntityFacts]` **Returns:** `EntityFacts` object containing all XBRL facts **Example:** `facts = company.get_facts() if facts: # Convert to pandas DataFrame facts_df = facts.to_pandas() # Get number of facts num_facts = facts.num_facts() print(f"Company has {num_facts} XBRL facts")` Address Information ------------------- ### business\_address() Get the company's business address. `def business_address(self) -> Optional[Address]` **Returns:** `Address` object or None **Example:** `address = company.business_address() if address: print(f"{address.street1}") print(f"{address.city}, {address.state_or_country} {address.zipcode}")` ### mailing\_address() Get the company's mailing address. `def mailing_address(self) -> Optional[Address]` **Returns:** `Address` object or None Utility Methods --------------- ### get\_ticker() Get the primary ticker symbol for the company. `def get_ticker(self) -> Optional[str]` **Returns:** Primary ticker symbol or None **Example:** `ticker = company.get_ticker() print(ticker) # "AAPL"` ### get\_exchanges() Get all exchanges where the company's stock is traded. `def get_exchanges(self) -> List[str]` **Returns:** List of exchange names **Example:** `exchanges = company.get_exchanges() print(exchanges) # ["NASDAQ"]` ### get\_icon() Get company icon (if available). `def get_icon(self)` **Returns:** Icon data or placeholder Data Access ----------- ### data Access the underlying company data object. `@property def data(self) -> EntityData` **Returns:** `EntityData` object with complete company information **Example:** `# Access detailed company data company_data = company.data print(company_data.former_names) # Previous company names print(company_data.entity_type) # Entity type print(company_data.flags) # SEC flags` Related Classes --------------- ### EntityFilings Collection of SEC filings returned by `get_filings()`. `filings = company.get_filings(form="10-K") # Collection methods latest = filings.latest() # Get latest filing first_five = filings.head(5) # Get first 5 filings random_sample = filings.sample(3) # Get 3 random filings # Filtering recent = filings.filter(filing_date="2023-01-01:") xbrl_only = filings.filter(is_xbrl=True) # Indexing first_filing = filings[0] # Get first filing second_filing = filings[1] # Get second filing # Iteration for filing in filings: print(f"{filing.form}: {filing.filing_date}") # Conversion filings_df = filings.to_pandas() # Convert to DataFrame` ### Address Physical address representation. `class Address: street1: str street2: Optional[str] city: str state_or_country: str zipcode: str state_or_country_desc: str` **Example:** `address = company.business_address() full_address = f"{address.street1}, {address.city}, {address.state_or_country}"` ### EntityFacts XBRL facts data container. `facts = company.get_facts() # Convert to DataFrame df = facts.to_pandas() # Get fact count count = facts.num_facts()` Factory Functions ----------------- Alternative ways to create Company instances: `from edgar import get_company, get_entity # Factory function company = get_company("AAPL") # More general entity function (returns Company for companies) entity = get_entity("AAPL")` Import Options -------------- `# Primary import from edgar import Company # Alternative imports from edgar.entity import Company from edgar.entity.core import Company` Error Handling -------------- `try: company = Company("INVALID") except CompanyNotFoundError: print("Company not found") except ValueError as e: print(f"Invalid identifier: {e}") # Check if company was found company = Company("MAYBE_INVALID") if company.not_found: print("Company data not available") else: filings = company.get_filings()` Performance Tips ---------------- 1. **Use CIK when possible** - faster than ticker lookup 2. **Cache Company objects** - avoid repeated API calls 3. **Filter filings efficiently** - use specific parameters in `get_filings()` 4. **Limit result sets** - use reasonable date ranges and form filters `# Efficient: specific filtering recent_10k = company.get_filings(form="10-K", filing_date="2023-01-01:") # Less efficient: get all then filter all_filings = company.get_filings() filtered = all_filings.filter(form="10-K").filter(filing_date="2023-01-01:")` Complete Example ---------------- `from edgar import Company # Create company instance company = Company("AAPL") # Basic information print(f"Company: {company.name}") print(f"CIK: {company.cik}") print(f"Industry: {company.industry}") print(f"Fiscal Year End: {company.fiscal_year_end}") # Key metrics — simple property access print(f"Shares Outstanding: {company.shares_outstanding:,.0f}") print(f"Public Float: ${company.public_float:,.0f}") # Get recent filings recent_filings = company.get_filings( form=["10-K", "10-Q"], filing_date="2023-01-01:", limit=5 ) print(f"\nRecent Filings ({len(recent_filings)}):") for filing in recent_filings: print(f" {filing.form}: {filing.filing_date}") # Get financial data financials = company.get_financials() if financials: revenue = financials.income.loc['Revenue'].iloc[0] print(f"\nLatest Revenue: ${revenue/1e9:.1f}B") # Get company facts facts = company.get_facts() if facts: print(f"Total XBRL Facts: {facts.num_facts()}") # Address information address = company.business_address() if address: print(f"Location: {address.city}, {address.state_or_country}")` See Also -------- * **[Finding Companies Guide](https://edgartools.readthedocs.io/en/latest/guides/finding-companies/) ** - How to locate companies * **[Filing API Reference](https://edgartools.readthedocs.io/en/latest/api/filing/) ** - Working with individual filings * **[Filings API Reference](https://edgartools.readthedocs.io/en/latest/api/filings/) ** - Working with filing collections * **[Extract Financial Statements](https://edgartools.readthedocs.io/en/latest/guides/extract-statements/) ** - Getting financial data Back to top --- # Company - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/api/company/#company-api-reference) Company API Reference ===================== The `Company` class is the primary interface for working with public companies in EdgarTools. It provides access to company information, SEC filings, and financial data. Class Overview -------------- `from edgar import Company class Company(Entity): """Represents a public company with SEC filings."""` **Inheritance:** `SecFiler` → `Entity` → `Company` Constructor ----------- ### Company(cik\_or\_ticker) Create a Company instance using either a CIK number or ticker symbol. `Company(cik_or_ticker: Union[str, int])` **Parameters:** - `cik_or_ticker` (Union\[str, int\]): Company identifier - **CIK**: Central Index Key as integer or string (with or without padding) - **Ticker**: Stock ticker symbol (case-insensitive) **Examples:** `# By ticker symbol (case-insensitive) company = Company("AAPL") company = Company("aapl") # By CIK number company = Company(320193) company = Company("320193") company = Company("0000320193") # Zero-padded` **Raises:** - `CompanyNotFoundError`: When company cannot be found - `ValueError`: When identifier format is invalid Core Properties --------------- ### Basic Information #### name `@property def name(self) -> str` Official company name as registered with the SEC. `company = Company("AAPL") print(company.name) # "Apple Inc."` #### cik `@property def cik(self) -> int` Central Index Key - unique identifier assigned by the SEC. `print(company.cik) # 320193` #### display\_name `@property def display_name(self) -> str` Formatted display name combining ticker and company name. `print(company.display_name) # "AAPL - Apple Inc."` #### tickers `@property def tickers(self) -> List[str]` List of all ticker symbols associated with the company. `berkshire = Company("BRK-A") print(berkshire.tickers) # ["BRK-A", "BRK-B"]` ### Industry & Classification #### industry `@property def industry(self) -> str` Industry description based on SIC code. `print(company.industry) # "ELECTRONIC COMPUTERS"` #### sic `@property def sic(self) -> str` Standard Industrial Classification code. `print(company.sic) # "3571"` #### fiscal\_year\_end `@property def fiscal_year_end(self) -> str` Fiscal year end date in MMDD format. `print(company.fiscal_year_end) # "0930" (September 30)` ### Company Status #### is\_company `@property def is_company(self) -> bool` Always `True` for Company instances. Used to distinguish from other entities. `print(company.is_company) # True` #### not\_found `@property def not_found(self) -> bool` Whether the company data was found in SEC database. `print(company.not_found) # False if found, True if not` ### Key Metrics #### shares\_outstanding `@property def shares_outstanding(self) -> Optional[float]` Number of common shares outstanding, sourced from SEC company facts. `company = Company("AAPL") print(company.shares_outstanding) # 15115785000.0` #### public\_float `@property def public_float(self) -> Optional[float]` Public float value in dollars, sourced from SEC company facts. `company = Company("AAPL") print(company.public_float) # 2899948348000.0` Filing Access ------------- ### get\_filings() Get company filings with extensive filtering options. `def get_filings( self, *, year: Union[int, List[int], range] = None, quarter: Union[int, List[int]] = None, form: Union[str, List[str]] = None, accession_number: Union[str, List[str]] = None, file_number: Union[str, List[str]] = None, filing_date: str = None, date: str = None, amendments: bool = True, is_xbrl: bool = None, is_inline_xbrl: bool = None, sort_by: str = "filing_date", trigger_full_load: bool = False ) -> EntityFilings` **Parameters:** - `year`: Filter by year(s) - int, list of ints, or range - `quarter`: Filter by quarter(s) - 1, 2, 3, or 4 - `form`: SEC form type(s) - e.g., "10-K", \["10-K", "10-Q"\] - `accession_number`: Specific accession number(s) - `file_number`: SEC file number(s) - `filing_date`: Date or date range (YYYY-MM-DD or YYYY-MM-DD:YYYY-MM-DD) - `date`: Alias for filing\_date - `amendments`: Include amended filings (default: True) - `is_xbrl`: Filter for XBRL filings - `is_inline_xbrl`: Filter for inline XBRL filings - `sort_by`: Sort field (default: "filing\_date") - `trigger_full_load`: Load all filing details upfront **Returns:** `EntityFilings` - Collection of company filings **Examples:** `# Get all filings all_filings = company.get_filings() # Get specific form types annual_reports = company.get_filings(form="10-K") quarterly_reports = company.get_filings(form=["10-K", "10-Q"]) # Filter by date recent = company.get_filings(filing_date="2023-01-01:") date_range = company.get_filings(filing_date="2023-01-01:2023-12-31") # Filter by year and quarter q4_2023 = company.get_filings(year=2023, quarter=4) multi_year = company.get_filings(year=[2022, 2023]) # XBRL filings only xbrl_filings = company.get_filings(is_xbrl=True) # Exclude amendments original_only = company.get_filings(amendments=False)` ### latest() Get the latest filing(s) of a specific form type. `def latest(self, form: str, n: int = 1) -> Union[Filing, List[Filing]]` **Parameters:** - `form`: SEC form type (e.g., "10-K", "10-Q", "8-K") - `n`: Number of latest filings to return (default: 1) **Returns:** - Single `Filing` if n=1 - `List[Filing]` if n>1 **Examples:** `# Get latest 10-K latest_10k = company.latest("10-K") # Get latest 3 quarterly reports latest_10qs = company.latest("10-Q", 3)` ### Convenience Properties #### latest\_tenk `@property def latest_tenk(self) -> Optional[TenK]` Latest 10-K filing as a TenK object with enhanced functionality. `tenk = company.latest_tenk if tenk: print(tenk.filing_date) financials = tenk.financials` #### latest\_tenq `@property def latest_tenq(self) -> Optional[TenQ]` Latest 10-Q filing as a TenQ object with enhanced functionality. `tenq = company.latest_tenq if tenq: print(tenq.filing_date) financials = tenq.financials` Financial Data -------------- ### get\_financials() Get financial statements from the latest 10-K filing. `def get_financials(self) -> Optional[Financials]` **Returns:** `Financials` object with balance sheet, income statement, and cash flow data **Example:** `financials = company.get_financials() if financials: balance_sheet = financials.balance_sheet income_statement = financials.income cash_flow = financials.cash_flow # Access specific metrics revenue = income_statement.loc['Revenue'].iloc[0] total_assets = balance_sheet.loc['Total Assets'].iloc[0]` ### get\_quarterly\_financials() Get financial statements from the latest 10-Q filing. `def get_quarterly_financials(self) -> Optional[Financials]` **Returns:** `Financials` object from latest quarterly report **Example:** `quarterly = company.get_quarterly_financials() if quarterly: q_income = quarterly.income quarterly_revenue = q_income.loc['Revenue'].iloc[0]` ### get\_facts() Get structured XBRL facts for the company. `def get_facts(self) -> Optional[EntityFacts]` **Returns:** `EntityFacts` object containing all XBRL facts **Example:** `facts = company.get_facts() if facts: # Convert to pandas DataFrame facts_df = facts.to_pandas() # Get number of facts num_facts = facts.num_facts() print(f"Company has {num_facts} XBRL facts")` Address Information ------------------- ### business\_address() Get the company's business address. `def business_address(self) -> Optional[Address]` **Returns:** `Address` object or None **Example:** `address = company.business_address() if address: print(f"{address.street1}") print(f"{address.city}, {address.state_or_country} {address.zipcode}")` ### mailing\_address() Get the company's mailing address. `def mailing_address(self) -> Optional[Address]` **Returns:** `Address` object or None Utility Methods --------------- ### get\_ticker() Get the primary ticker symbol for the company. `def get_ticker(self) -> Optional[str]` **Returns:** Primary ticker symbol or None **Example:** `ticker = company.get_ticker() print(ticker) # "AAPL"` ### get\_exchanges() Get all exchanges where the company's stock is traded. `def get_exchanges(self) -> List[str]` **Returns:** List of exchange names **Example:** `exchanges = company.get_exchanges() print(exchanges) # ["NASDAQ"]` ### get\_icon() Get company icon (if available). `def get_icon(self)` **Returns:** Icon data or placeholder Data Access ----------- ### data Access the underlying company data object. `@property def data(self) -> EntityData` **Returns:** `EntityData` object with complete company information **Example:** `# Access detailed company data company_data = company.data print(company_data.former_names) # Previous company names print(company_data.entity_type) # Entity type print(company_data.flags) # SEC flags` Related Classes --------------- ### EntityFilings Collection of SEC filings returned by `get_filings()`. `filings = company.get_filings(form="10-K") # Collection methods latest = filings.latest() # Get latest filing first_five = filings.head(5) # Get first 5 filings random_sample = filings.sample(3) # Get 3 random filings # Filtering recent = filings.filter(filing_date="2023-01-01:") xbrl_only = filings.filter(is_xbrl=True) # Indexing first_filing = filings[0] # Get first filing second_filing = filings[1] # Get second filing # Iteration for filing in filings: print(f"{filing.form}: {filing.filing_date}") # Conversion filings_df = filings.to_pandas() # Convert to DataFrame` ### Address Physical address representation. `class Address: street1: str street2: Optional[str] city: str state_or_country: str zipcode: str state_or_country_desc: str` **Example:** `address = company.business_address() full_address = f"{address.street1}, {address.city}, {address.state_or_country}"` ### EntityFacts XBRL facts data container. `facts = company.get_facts() # Convert to DataFrame df = facts.to_pandas() # Get fact count count = facts.num_facts()` Factory Functions ----------------- Alternative ways to create Company instances: `from edgar import get_company, get_entity # Factory function company = get_company("AAPL") # More general entity function (returns Company for companies) entity = get_entity("AAPL")` Import Options -------------- `# Primary import from edgar import Company # Alternative imports from edgar.entity import Company from edgar.entity.core import Company` Error Handling -------------- `try: company = Company("INVALID") except CompanyNotFoundError: print("Company not found") except ValueError as e: print(f"Invalid identifier: {e}") # Check if company was found company = Company("MAYBE_INVALID") if company.not_found: print("Company data not available") else: filings = company.get_filings()` Performance Tips ---------------- 1. **Use CIK when possible** - faster than ticker lookup 2. **Cache Company objects** - avoid repeated API calls 3. **Filter filings efficiently** - use specific parameters in `get_filings()` 4. **Limit result sets** - use reasonable date ranges and form filters `# Efficient: specific filtering recent_10k = company.get_filings(form="10-K", filing_date="2023-01-01:") # Less efficient: get all then filter all_filings = company.get_filings() filtered = all_filings.filter(form="10-K").filter(filing_date="2023-01-01:")` Complete Example ---------------- `from edgar import Company # Create company instance company = Company("AAPL") # Basic information print(f"Company: {company.name}") print(f"CIK: {company.cik}") print(f"Industry: {company.industry}") print(f"Fiscal Year End: {company.fiscal_year_end}") # Key metrics — simple property access print(f"Shares Outstanding: {company.shares_outstanding:,.0f}") print(f"Public Float: ${company.public_float:,.0f}") # Get recent filings recent_filings = company.get_filings( form=["10-K", "10-Q"], filing_date="2023-01-01:", limit=5 ) print(f"\nRecent Filings ({len(recent_filings)}):") for filing in recent_filings: print(f" {filing.form}: {filing.filing_date}") # Get financial data financials = company.get_financials() if financials: revenue = financials.income.loc['Revenue'].iloc[0] print(f"\nLatest Revenue: ${revenue/1e9:.1f}B") # Get company facts facts = company.get_facts() if facts: print(f"Total XBRL Facts: {facts.num_facts()}") # Address information address = company.business_address() if address: print(f"Location: {address.city}, {address.state_or_country}")` See Also -------- * **[Finding Companies Guide](https://edgartools.readthedocs.io/en/stable/guides/finding-companies/) ** - How to locate companies * **[Filing API Reference](https://edgartools.readthedocs.io/en/stable/api/filing/) ** - Working with individual filings * **[Filings API Reference](https://edgartools.readthedocs.io/en/stable/api/filings/) ** - Working with filing collections * **[Extract Financial Statements](https://edgartools.readthedocs.io/en/stable/guides/extract-statements/) ** - Getting financial data Back to top --- # Find a Company - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/finding-companies/#find-sec-companies-by-ticker-cik-or-name) Find SEC Companies by Ticker, CIK, or Name ========================================== Learn how to locate companies in the SEC database using tickers, CIKs, or company names. Method 1: Find by Ticker Symbol ------------------------------- The most common way to find a company is by its stock ticker symbol: `from edgar import Company # Find Apple by ticker (case-insensitive) apple = Company("AAPL") print(apple)` ![Company lookup output](https://edgartools.readthedocs.io/en/latest/images/company-lookup.webp) **Key points:** - Tickers are case-insensitive: `Company("aapl")` works the same as `Company("AAPL")` - This performs a ticker lookup then loads the company data - Some companies have multiple tickers for the same entity Method 2: Find by CIK (Central Index Key) ----------------------------------------- The CIK uniquely identifies every SEC filer and is more reliable than tickers: `# Using numeric CIK apple = Company(320193) # Using string CIK (with or without zero padding) apple = Company("320193") apple = Company("0000320193") print(apple)` **Why use CIK:** - **Unique**: Every company has exactly one CIK - **Permanent**: CIKs don't change like tickers might - **Faster**: Direct lookup without ticker resolution Method 3: Search by Company Name -------------------------------- When you don't know the exact ticker or CIK: `from edgar import find # Search for companies by name results = find("Apple") print(f"Found {len(results)} companies:") for company in results: print(f" {company.ticker}: {company.name}")` **Output:** `Found 3 companies: AAPL: Apple Inc. APPL: Apple Hospitality REIT Inc APOG: Apogee Enterprises Inc` **Then select the right one:** `# Get the first result apple = results[0] # Or be more specific apple = Company("AAPL") # If you know the ticker from search` Working with Company Objects ---------------------------- Once you have a Company object, you can access detailed information: `company = Company("MSFT") # Basic information print(f"Name: {company.name}") print(f"CIK: {company.cik}") print(f"Ticker: {company.ticker}") print(f"Industry: {company.industry}") print(f"Website: {company.website}") print(f"Location: {company.city}, {company.state}") # SEC-specific information print(f"SIC Code: {company.sic}") print(f"Fiscal Year End: {company.fiscal_year_end}") print(f"Exchange: {company.exchange}")` **Output:** `Name: Microsoft Corporation CIK: 0000789019 Ticker: MSFT Industry: SERVICES-PREPACKAGED SOFTWARE Website: https://www.microsoft.com Location: Redmond, WA SIC Code: 7372 Fiscal Year End: 0630 Exchange: Nasdaq` Handling Edge Cases ------------------- ### Company Not Found `try: company = Company("INVALID") except Exception as e: print(f"Company not found: {e}") # Fallback to search results = find("Invalid Corp") if results: company = results[0] else: print("No companies found matching that name")` ### Multiple Tickers for Same Company `# Berkshire Hathaway has multiple share classes brk_a = Company("BRK-A") # Class A shares brk_b = Company("BRK-B") # Class B shares # Both point to the same CIK and SEC filings print(f"BRK-A CIK: {brk_a.cik}") print(f"BRK-B CIK: {brk_b.cik}") # Both will show: 0001067983` ### Historical Tickers `# Some companies change tickers over time # The Company object will find the current entity try: company = Company("FB") # Meta's old ticker print(f"Found: {company.name}") # May find Meta Platforms Inc except: # Try the new ticker company = Company("META") print(f"Found: {company.name}")` Batch Company Lookup -------------------- For analyzing multiple companies efficiently: `from edgar import Company import pandas as pd tickers = ["AAPL", "MSFT", "GOOGL", "AMZN", "META"] companies = [] for ticker in tickers: try: company = Company(ticker) companies.append({ 'ticker': ticker, 'name': company.name, 'cik': company.cik, 'industry': company.industry, }) print(f"Found {ticker}: {company.name}") except Exception as e: print(f"Error with {ticker}: {e}") df = pd.DataFrame(companies) print(df)` ![Batch company lookup DataFrame](https://edgartools.readthedocs.io/en/latest/images/company-batch-lookup.webp) Company Screening with Shares Outstanding & Public Float -------------------------------------------------------- Every public company reports **shares outstanding** and **public float** to the SEC. EdgarTools gives you direct access to these as simple properties: `company = Company("AAPL") # Shares outstanding — total common shares issued print(f"Shares Outstanding: {company.shares_outstanding:,.0f}") # 15,115,785,000 # Public float — dollar value of shares available for public trading print(f"Public Float: ${company.public_float:,.0f}") # $2,899,948,348,000` ### Screen Multiple Companies Build a screening table across any set of companies: `import pandas as pd from edgar import Company tickers = ["AAPL", "MSFT", "NVDA", "AMZN", "META", "TSLA"] rows = [] for ticker in tickers: company = Company(ticker) rows.append({ 'ticker': ticker, 'name': company.name, 'industry': company.industry, 'shares_outstanding': company.shares_outstanding, 'public_float': company.public_float, }) df = pd.DataFrame(rows) # Sort by public float (largest first) df = df.sort_values('public_float', ascending=False) print(df.to_string(index=False))` ![Company screening with shares outstanding and public float](https://edgartools.readthedocs.io/en/latest/images/company-screening.webp) ### Filter by Float Size `# Find mega-cap companies (float > $1 trillion) mega_caps = df[df['public_float'] > 1e12] print(f"Mega-cap companies: {list(mega_caps['ticker'])}") # Find companies with low share count (< 1 billion shares) low_share_count = df[df['shares_outstanding'] < 1e9] print(f"Low share count: {list(low_share_count['ticker'])}")` Advanced Search Techniques -------------------------- ### Search by Industry Use the reference module to find companies by SIC industry code: `from edgar.reference import get_companies_by_industry # Get all software companies (SIC 7372) software = get_companies_by_industry(sic=7372) print(f"Found {len(software)} software companies") print(software.head())` ![Search companies by industry (SIC code)](https://edgartools.readthedocs.io/en/latest/images/company-search-by-industry.webp) ### Search by Exchange `from edgar.reference import get_companies_by_exchanges # Get all NYSE-listed companies nyse = get_companies_by_exchanges("NYSE") print(f"NYSE companies: {len(nyse)}") print(nyse.head()) # Get Nasdaq companies nasdaq = get_companies_by_exchanges("Nasdaq") print(f"Nasdaq companies: {len(nasdaq)}")` ![Search companies by exchange](https://edgartools.readthedocs.io/en/latest/images/company-search-by-exchange.webp) ### Search by State `from edgar.reference import get_companies_by_state # Get all companies incorporated in Delaware delaware = get_companies_by_state("DE") print(f"Delaware companies: {len(delaware)}")` ### Search by Filing Activity `from edgar import get_filings, Company import pandas as pd # Find companies that filed 8-K forms recently recent_8k_filings = get_filings(form="8-K") active_companies = [] for filing in recent_8k_filings.head(50): try: company = Company(filing.cik) active_companies.append({ 'ticker': company.ticker, 'name': company.name, 'filing_date': filing.filing_date, 'cik': company.cik, }) except Exception: continue df = pd.DataFrame(active_companies) recent_activity = df.sort_values('filing_date', ascending=False).head(10) print(recent_activity)` How Ticker Resolution Works --------------------------- When you call `Company("AAPL")`, edgartools resolves the ticker to a CIK number using a three-level waterfall: 1. **Bundled data (instant, no network)** — edgartools ships with a `company_tickers.parquet` file containing ~10,600 exchange-listed tickers. This is tried first and works completely offline. 2. **Local downloaded data** — If you've called `download_edgar_data(reference=True)` and enabled `use_local_storage()`, locally cached ticker data is used. This includes the full SEC ticker universe. 3. **Live SEC API** — As a final fallback for brand-new tickers (e.g., recent IPOs not yet in bundled data), edgartools fetches from `data.sec.gov`. This means **`Company()` lookups work offline by default** for established tickers — no setup required. For full offline coverage including recent IPOs, see the [Local Storage guide](https://edgartools.readthedocs.io/en/latest/guides/local-storage/) . `# Works offline — no internet needed for established tickers company = Company("AAPL") # For full offline coverage (including recent IPOs): from edgar import download_edgar_data, use_local_storage download_edgar_data(submissions=False, facts=False, reference=True) use_local_storage()` Performance Tips ---------------- 1. **Use CIK when possible**: Faster than ticker lookup 2. **Cache company objects**: If analyzing the same companies repeatedly 3. **Batch processing**: Handle errors gracefully in loops 4. **Check data availability**: Not all companies have all fields populated Common Issues ------------- ### Ticker vs Company Name Confusion `# This will fail - searching for ticker in name search results = find("AAPL") # Returns companies with "AAPL" in name, not ticker # Use Company() for ticker lookup company = Company("AAPL") # Correct for ticker lookup` ### International Companies `# Some foreign companies trade on US exchanges try: company = Company("ASML") # Dutch company on NASDAQ print(f"Found: {company.name} in {company.country}") except: print("Company not found or not SEC-registered")` ### Delisted Companies `# Some companies may be delisted but still have SEC filings try: company = Company("1234567") # Use CIK for delisted companies print(f"Company: {company.name}") print(f"Status: {'Active' if company.ticker else 'Possibly delisted'}") except: print("Company not found in SEC database")` Next Steps ---------- Now that you can find and screen companies, learn how to: * **[Company Facts & Financial Data](https://edgartools.readthedocs.io/en/latest/guides/company-facts/) ** - Shares outstanding, public float, and financial statements * **[Search for Specific Filings](https://edgartools.readthedocs.io/en/latest/guides/searching-filings/) ** - Find the documents you need * **[Extract Financial Statements](https://edgartools.readthedocs.io/en/latest/guides/extract-statements/) ** - Get financial data * **[Filter Filings by Date/Type](https://edgartools.readthedocs.io/en/latest/guides/filtering-filings/) ** - Narrow down your search Related Documentation --------------------- * **[Company API Reference](https://edgartools.readthedocs.io/en/latest/api/company/) ** - Complete Company class documentation * **[Business Overview Data Sources](https://edgartools.readthedocs.io/en/latest/guides/business-overview-data-sources-guide/) ** - Build company overview pages * **[Company Subsets](https://edgartools.readthedocs.io/en/latest/company-subsets/) ** - Create research datasets by exchange, industry, or popularity * **[Local Storage](https://edgartools.readthedocs.io/en/latest/guides/local-storage/) ** - Offline usage and caching ticker data locally Back to top --- # Find a Company - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/finding-companies/#find-sec-companies-by-ticker-cik-or-name) Find SEC Companies by Ticker, CIK, or Name ========================================== Learn how to locate companies in the SEC database using tickers, CIKs, or company names. Method 1: Find by Ticker Symbol ------------------------------- The most common way to find a company is by its stock ticker symbol: `from edgar import Company # Find Apple by ticker (case-insensitive) apple = Company("AAPL") print(apple)` ![Company lookup output](https://edgartools.readthedocs.io/en/stable/images/company-lookup.webp) **Key points:** - Tickers are case-insensitive: `Company("aapl")` works the same as `Company("AAPL")` - This performs a ticker lookup then loads the company data - Some companies have multiple tickers for the same entity Method 2: Find by CIK (Central Index Key) ----------------------------------------- The CIK uniquely identifies every SEC filer and is more reliable than tickers: `# Using numeric CIK apple = Company(320193) # Using string CIK (with or without zero padding) apple = Company("320193") apple = Company("0000320193") print(apple)` **Why use CIK:** - **Unique**: Every company has exactly one CIK - **Permanent**: CIKs don't change like tickers might - **Faster**: Direct lookup without ticker resolution Method 3: Search by Company Name -------------------------------- When you don't know the exact ticker or CIK: `from edgar import find # Search for companies by name results = find("Apple") print(f"Found {len(results)} companies:") for company in results: print(f" {company.ticker}: {company.name}")` **Output:** `Found 3 companies: AAPL: Apple Inc. APPL: Apple Hospitality REIT Inc APOG: Apogee Enterprises Inc` **Then select the right one:** `# Get the first result apple = results[0] # Or be more specific apple = Company("AAPL") # If you know the ticker from search` Working with Company Objects ---------------------------- Once you have a Company object, you can access detailed information: `company = Company("MSFT") # Basic information print(f"Name: {company.name}") print(f"CIK: {company.cik}") print(f"Ticker: {company.ticker}") print(f"Industry: {company.industry}") print(f"Website: {company.website}") print(f"Location: {company.city}, {company.state}") # SEC-specific information print(f"SIC Code: {company.sic}") print(f"Fiscal Year End: {company.fiscal_year_end}") print(f"Exchange: {company.exchange}")` **Output:** `Name: Microsoft Corporation CIK: 0000789019 Ticker: MSFT Industry: SERVICES-PREPACKAGED SOFTWARE Website: https://www.microsoft.com Location: Redmond, WA SIC Code: 7372 Fiscal Year End: 0630 Exchange: Nasdaq` Handling Edge Cases ------------------- ### Company Not Found `try: company = Company("INVALID") except Exception as e: print(f"Company not found: {e}") # Fallback to search results = find("Invalid Corp") if results: company = results[0] else: print("No companies found matching that name")` ### Multiple Tickers for Same Company `# Berkshire Hathaway has multiple share classes brk_a = Company("BRK-A") # Class A shares brk_b = Company("BRK-B") # Class B shares # Both point to the same CIK and SEC filings print(f"BRK-A CIK: {brk_a.cik}") print(f"BRK-B CIK: {brk_b.cik}") # Both will show: 0001067983` ### Historical Tickers `# Some companies change tickers over time # The Company object will find the current entity try: company = Company("FB") # Meta's old ticker print(f"Found: {company.name}") # May find Meta Platforms Inc except: # Try the new ticker company = Company("META") print(f"Found: {company.name}")` Batch Company Lookup -------------------- For analyzing multiple companies efficiently: `from edgar import Company import pandas as pd tickers = ["AAPL", "MSFT", "GOOGL", "AMZN", "META"] companies = [] for ticker in tickers: try: company = Company(ticker) companies.append({ 'ticker': ticker, 'name': company.name, 'cik': company.cik, 'industry': company.industry, }) print(f"Found {ticker}: {company.name}") except Exception as e: print(f"Error with {ticker}: {e}") df = pd.DataFrame(companies) print(df)` ![Batch company lookup DataFrame](https://edgartools.readthedocs.io/en/stable/images/company-batch-lookup.webp) Company Screening with Shares Outstanding & Public Float -------------------------------------------------------- Every public company reports **shares outstanding** and **public float** to the SEC. EdgarTools gives you direct access to these as simple properties: `company = Company("AAPL") # Shares outstanding — total common shares issued print(f"Shares Outstanding: {company.shares_outstanding:,.0f}") # 15,115,785,000 # Public float — dollar value of shares available for public trading print(f"Public Float: ${company.public_float:,.0f}") # $2,899,948,348,000` ### Screen Multiple Companies Build a screening table across any set of companies: `import pandas as pd from edgar import Company tickers = ["AAPL", "MSFT", "NVDA", "AMZN", "META", "TSLA"] rows = [] for ticker in tickers: company = Company(ticker) rows.append({ 'ticker': ticker, 'name': company.name, 'industry': company.industry, 'shares_outstanding': company.shares_outstanding, 'public_float': company.public_float, }) df = pd.DataFrame(rows) # Sort by public float (largest first) df = df.sort_values('public_float', ascending=False) print(df.to_string(index=False))` ![Company screening with shares outstanding and public float](https://edgartools.readthedocs.io/en/stable/images/company-screening.webp) ### Filter by Float Size `# Find mega-cap companies (float > $1 trillion) mega_caps = df[df['public_float'] > 1e12] print(f"Mega-cap companies: {list(mega_caps['ticker'])}") # Find companies with low share count (< 1 billion shares) low_share_count = df[df['shares_outstanding'] < 1e9] print(f"Low share count: {list(low_share_count['ticker'])}")` Advanced Search Techniques -------------------------- ### Search by Industry Use the reference module to find companies by SIC industry code: `from edgar.reference import get_companies_by_industry # Get all software companies (SIC 7372) software = get_companies_by_industry(sic=7372) print(f"Found {len(software)} software companies") print(software.head())` ![Search companies by industry (SIC code)](https://edgartools.readthedocs.io/en/stable/images/company-search-by-industry.webp) ### Search by Exchange `from edgar.reference import get_companies_by_exchanges # Get all NYSE-listed companies nyse = get_companies_by_exchanges("NYSE") print(f"NYSE companies: {len(nyse)}") print(nyse.head()) # Get Nasdaq companies nasdaq = get_companies_by_exchanges("Nasdaq") print(f"Nasdaq companies: {len(nasdaq)}")` ![Search companies by exchange](https://edgartools.readthedocs.io/en/stable/images/company-search-by-exchange.webp) ### Search by State `from edgar.reference import get_companies_by_state # Get all companies incorporated in Delaware delaware = get_companies_by_state("DE") print(f"Delaware companies: {len(delaware)}")` ### Search by Filing Activity `from edgar import get_filings, Company import pandas as pd # Find companies that filed 8-K forms recently recent_8k_filings = get_filings(form="8-K") active_companies = [] for filing in recent_8k_filings.head(50): try: company = Company(filing.cik) active_companies.append({ 'ticker': company.ticker, 'name': company.name, 'filing_date': filing.filing_date, 'cik': company.cik, }) except Exception: continue df = pd.DataFrame(active_companies) recent_activity = df.sort_values('filing_date', ascending=False).head(10) print(recent_activity)` How Ticker Resolution Works --------------------------- When you call `Company("AAPL")`, edgartools resolves the ticker to a CIK number using a three-level waterfall: 1. **Bundled data (instant, no network)** — edgartools ships with a `company_tickers.parquet` file containing ~10,600 exchange-listed tickers. This is tried first and works completely offline. 2. **Local downloaded data** — If you've called `download_edgar_data(reference=True)` and enabled `use_local_storage()`, locally cached ticker data is used. This includes the full SEC ticker universe. 3. **Live SEC API** — As a final fallback for brand-new tickers (e.g., recent IPOs not yet in bundled data), edgartools fetches from `data.sec.gov`. This means **`Company()` lookups work offline by default** for established tickers — no setup required. For full offline coverage including recent IPOs, see the [Local Storage guide](https://edgartools.readthedocs.io/en/stable/guides/local-storage/) . `# Works offline — no internet needed for established tickers company = Company("AAPL") # For full offline coverage (including recent IPOs): from edgar import download_edgar_data, use_local_storage download_edgar_data(submissions=False, facts=False, reference=True) use_local_storage()` Performance Tips ---------------- 1. **Use CIK when possible**: Faster than ticker lookup 2. **Cache company objects**: If analyzing the same companies repeatedly 3. **Batch processing**: Handle errors gracefully in loops 4. **Check data availability**: Not all companies have all fields populated Common Issues ------------- ### Ticker vs Company Name Confusion `# This will fail - searching for ticker in name search results = find("AAPL") # Returns companies with "AAPL" in name, not ticker # Use Company() for ticker lookup company = Company("AAPL") # Correct for ticker lookup` ### International Companies `# Some foreign companies trade on US exchanges try: company = Company("ASML") # Dutch company on NASDAQ print(f"Found: {company.name} in {company.country}") except: print("Company not found or not SEC-registered")` ### Delisted Companies `# Some companies may be delisted but still have SEC filings try: company = Company("1234567") # Use CIK for delisted companies print(f"Company: {company.name}") print(f"Status: {'Active' if company.ticker else 'Possibly delisted'}") except: print("Company not found in SEC database")` Next Steps ---------- Now that you can find and screen companies, learn how to: * **[Company Facts & Financial Data](https://edgartools.readthedocs.io/en/stable/guides/company-facts/) ** - Shares outstanding, public float, and financial statements * **[Search for Specific Filings](https://edgartools.readthedocs.io/en/stable/guides/searching-filings/) ** - Find the documents you need * **[Extract Financial Statements](https://edgartools.readthedocs.io/en/stable/guides/extract-statements/) ** - Get financial data * **[Filter Filings by Date/Type](https://edgartools.readthedocs.io/en/stable/guides/filtering-filings/) ** - Narrow down your search Related Documentation --------------------- * **[Company API Reference](https://edgartools.readthedocs.io/en/stable/api/company/) ** - Complete Company class documentation * **[Business Overview Data Sources](https://edgartools.readthedocs.io/en/stable/guides/business-overview-data-sources-guide/) ** - Build company overview pages * **[Company Subsets](https://edgartools.readthedocs.io/en/stable/company-subsets/) ** - Create research datasets by exchange, industry, or popularity * **[Local Storage](https://edgartools.readthedocs.io/en/stable/guides/local-storage/) ** - Offline usage and caching ticker data locally Back to top --- # Cheat Sheet - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/quick-guide/#cheat-sheet) Cheat Sheet =========== Common EdgarTools operations at a glance. For a step-by-step introduction, see the [Quick Start](https://edgartools.readthedocs.io/en/latest/quickstart/) . ### Setup | | Code | | --- | --- | | Set your EDGAR identity in Linux/Mac | `export EDGAR_IDENTITY="email@domain.com"` | | Set your EDGAR identity in Windows | `set EDGAR_IDENTITY="email@domain.com"` | | Set identity in Windows Powershell | `$env:EDGAR_IDENTITY="email@domain.com"` | | Set identity in Python | `set_identity("email@domain.com")` | | Importing the library | `from edgar import *` | ### Working with a company 🏢 > See also: [Find a Company](https://edgartools.readthedocs.io/en/latest/guides/finding-companies/) | | Code | | --- | --- | | 🔍 Get a company by ticker | `company = Company("AAPL")` | | 🔍 Get a company by CIK | `company = Company("0000320193")` | | 🔎 Find filings by form and ticker | `find(form="10-K", ticker="AAPL")` | | 📊 Get shares outstanding | `company.shares_outstanding` | | 💰 Get public float | `company.public_float` | | 🏭 Get industry | `company.industry` | | 📋 Get company facts | `company.get_facts()` | | 🐼 Get company facts as a DataFrame | `company.get_facts().to_pandas()` | ### Financial statements 💵 > See also: [Financial Statements Guide](https://edgartools.readthedocs.io/en/latest/guides/financial-data/) | | Code | | --- | --- | | 📊 Get a company's financials | `financials = company.get_financials()` | | 📈 Get the income statement | `financials.income_statement()` | | 🏦 Get the balance sheet | `financials.balance_sheet()` | | 💸 Get the cash flow statement | `financials.cashflow_statement()` | | 💰 Get revenue | `financials.get_revenue()` | | 💵 Get net income | `financials.get_net_income()` | | 📊 Get operating income | `financials.get_operating_income()` | | 🐼 Export statement to DataFrame | `financials.income_statement().to_dataframe()` | ### Working with filings 📁 > See also: [Working with Filings](https://edgartools.readthedocs.io/en/latest/guides/working-with-filing/) > · [Search & Filter](https://edgartools.readthedocs.io/en/latest/guides/searching-filings/) #### 🔍 Getting Filings | | Code | | --- | --- | | 📅 Get filings for the year to date | `filings = get_filings()` | | 📊 Get only XBRL filings | `filings = get_filings(index="xbrl")` | | 📆 Get filings for a specific year | `filings = get_filings(2020)` | | 🗓️ Get filings for a specific quarter | `filings = get_filings(2020, 1)` | | 📚 Get filings for multiple years | `filings = get_filings([2020, 2021])` | | 📈 Get filings for a range of years | `filings = get_filings(year=range(2010, 2020))` | | 📈 Get filings released just now | `filings = get_latest_filings()` | #### 📄 Filtering Filings | | Code | | --- | --- | | 📝 Filter by form type | `filings.filter(form="10-K")` | | 📑 Filter by multiple forms | `filings.filter(form=["10-K", "10-Q"])` | | 🔄 Include form amendments | `filings.filter(form="10-K", amendments=True)` | | 🏢 Filter by CIK | `filings.filter(cik="0000320193")` | | 🏙️ Filter by multiple CIKs | `filings.filter(cik=["0000320193", "1018724"])` | | 🏷️ Filter by ticker | `filings.filter(ticker="AAPL")` | | 🏷️🏷️ Filter by multiple tickers | `filings.filter(ticker=["AAPL", "MSFT"])` | | 📅 Filter on a specific date | `filings.filter(date="2020-01-01")` | | 📅↔️📅 Filter between dates | `filings.filter(date="2020-01-01:2020-03-01")` | | 📅⬅️ Filter before a date | `filings.filter(date=":2020-03-01")` | | 📅➡️ Filter after a date | `filings.filter(date="2020-03-01:")` | | 🔀 Combine multiple filters | `filings.filter(form="10-K", date="2020-01-01:", ticker="AAPL")` | #### 📊 Viewing and Manipulating Filings | | Code | | --- | --- | | ⏭️ Show the next page of filings | `filings.next()` | | ⏮️ Show the previous page of filings | `filings.previous()` | | 🔝 Get the first n filings | `filings.head(20)` | | 🔚 Get the last n filings | `filings.tail(20)` | | 🕒 Get the latest n filings by date | `filings.latest(20)` | | 🎲 Get a random sample of filings | `filings.sample(20)` | | 🐼 Get filings as a pandas DataFrame | `filings.to_pandas()` | ### Company filings 📂 > See also: [Find a Company](https://edgartools.readthedocs.io/en/latest/guides/finding-companies/) | | Code | | --- | --- | | 📁 Get company filings | `company.get_filings()` | | 📝 Get company filings by form | `company.get_filings(form="10-K")` | | 🕒 Get the latest 10-Q | `company.latest("10-Q")` | | 📑 Get the last 5 10-Qs | `company.get_filings(form="10-Q").head(5)` | | 🔢 Get a filing by accession number | `company.get_filing(accession_number="0000320193-21-000139")` | ### Working with a filing 📄 > See also: [Working with Filings](https://edgartools.readthedocs.io/en/latest/guides/working-with-filing/) #### 🔍 Accessing and Viewing a Filing | | Code | | --- | --- | | 📌 Get a single filing | `filing = filings[3]` | | 🔢 Get a filing by accession number | `filing = get_by_accession_number("0000320193-20-34576")` | | 🏠 Get the filing homepage | `filing.homepage` | | 🌐 Open a filing in the browser | `filing.open()` | | 🏠 Open homepage in the browser | `filing.homepage.open()` | | 💻 View the filing in the terminal | `filing.view()` | #### 📊 Extracting Filing Content | | Code | | --- | --- | | 🌐 Get the HTML of the filing | `filing.html()` | | 📊 Get the XBRL of the filing | `filing.xbrl()` | | 📝 Get the filing as markdown | `filing.markdown()` | | 📄 Get the full submission text | `filing.full_text_submission()` | | 🔍 Preview data object type | `filing.obj_type` | | 🔢 Get and parse filing data object | `filing.obj()` | | 📑 Get filing header | `filing.header` | #### 🔎 Searching Inside a Filing | | Code | | --- | --- | | 🔍 Search within the filing | `filing.search("query")` | | 🔍 Search with regex | `filing.search("pattern", regex=True)` | | 📊 Get filing sections | `filing.sections()` | #### 📎 Working with Attachments > See also: [Filing Attachments](https://edgartools.readthedocs.io/en/latest/guides/filing-attachments/) | | Code | | --- | --- | | 📁 Get all filing attachments | `filing.attachments` | | 📄 Get a single attachment | `attachment = filing.attachments[0]` | | 🌐 Open attachment in browser | `attachment.open()` | | ⬇️ Download an attachment | `content = attachment.download()` | ### 10-K Annual Report data 📊 > See also: [Working with Filings](https://edgartools.readthedocs.io/en/latest/guides/working-with-filing/) | | Code | | --- | --- | | 📄 Get 10-K as data object | `tenk = company.get_filings(form="10-K").latest().obj()` | | 🏢 Get auditor information | `tenk.auditor` | | 🏢 Get auditor name | `tenk.auditor.name` | | 🔢 Get PCAOB firm ID | `tenk.auditor.firm_id` | | 🏗️ Get subsidiaries | `tenk.subsidiaries` | | 🐼 Subsidiaries as DataFrame | `tenk.subsidiaries.to_dataframe()` | ### Proxy statements (executive compensation) 💼 > See also: [Proxy Statements Guide](https://edgartools.readthedocs.io/en/latest/guides/proxystatement-data-object-guide/) | | Code | | --- | --- | | 📋 Get latest proxy statement | `proxy = company.get_filings(form="DEF 14A").latest().obj()` | | 👤 Get CEO name | `proxy.peo_name` | | 💰 Get CEO total compensation | `proxy.peo_total_comp` | | 📊 Get 5-year exec compensation DataFrame | `proxy.executive_compensation` | | 📈 Get pay vs performance DataFrame | `proxy.pay_vs_performance` | | 📉 Get company TSR | `proxy.total_shareholder_return` | | 📉 Get peer group TSR | `proxy.peer_group_tsr` | Prefer a visual interface? Every operation above also works through **[edgar.tools](https://app.edgar.tools/?utm_source=edgartools-docs&utm_medium=see-live&utm_content=cheat-sheet) ** — the same SEC data in a web UI, no code required. * **[Browse any company's filings and financials →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=cheat-sheet) ** * **[Watch filings arrive in real time →](https://app.edgar.tools/filings?utm_source=edgartools-docs&utm_medium=see-live&utm_content=cheat-sheet) ** * **[Search disclosures across 12 XBRL topics →](https://app.edgar.tools/disclosures?utm_source=edgartools-docs&utm_medium=see-live&utm_content=cheat-sheet) ** Also includes a REST API (20+ endpoints), hosted MCP server, and data exports. Free tier: 100 API calls/day. Back to top --- # Choosing the Right API - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/xbrl/getting-started/choosing-the-right-api/#choosing-the-right-api) Choosing the Right API ====================== EdgarTools offers three different ways to access financial data. This guide helps you choose the right one for your needs. Quick Decision Tree ------------------- `What do you want to do? ├─ "Get historical financial trends for one company" │ └─> Use Company Facts API (company.income_statement()) │ ├─ "Compare metrics across multiple companies" │ └─> Use Financials API (company.get_financials()) │ ├─ "Need segment data, dimensions, or detailed breakdowns" │ └─> Use XBRL API (filing.xbrl().statements) │ └─ "Need footnotes or custom concepts" └─> Use XBRL API (filing.xbrl())` The Three APIs at a Glance -------------------------- ### 1\. Company Facts API - Simplest `company = Company("AAPL") income = company.income_statement() # Multi-year data instantly` ### 2\. Financials API - Best for Comparison `company = Company("AAPL") financials = company.get_financials() revenue = financials.get_revenue()` ### 3\. XBRL API - Most Complete `filing = Company("AAPL").get_filings(form="10-K").latest() xbrl = filing.xbrl() statements = xbrl.statements` Detailed Comparison ------------------- | Feature | Company Facts | Financials | XBRL | | --- | --- | --- | --- | | **Speed** | Fastest (cached) | Fast | Slower (parses filing) | | **Lines of code** | 1-2 | 2-3 | 3-5 | | **Multi-period data** | Built-in | Built-in | Manual filtering | | **Historical range** | All available periods | Recent filings | Single filing only | | **Statements** | Primary 3 only | Primary 3 only | All statements | | **Segment/dimension data** | No | No | Yes | | **Footnotes** | No | No | Yes | | **Custom concepts** | No | Limited | All concepts | | **Standardization** | Partial | Yes | Raw (you control) | | **Cross-company comparison** | Manual | Built-in | Manual | Use Case Examples ----------------- ### Scenario 1: "I want Apple's revenue for the last 5 years" **Recommended: Company Facts API** `from edgar import Company company = Company("AAPL") income = company.income_statement() # Get all revenue values revenues = income.get_all_values("Revenues") for value in revenues[:5]: print(f"{value.period}: ${value.value:,.0f}")` **Why this API?** - Single company, historical trend - Standard metric (revenue) - Fastest way to get multi-period data * * * ### Scenario 2: "Compare revenue growth: Apple vs Microsoft" **Recommended: Financials API** `from edgar import Company aapl = Company("AAPL").get_financials() msft = Company("MSFT").get_financials() print(f"Apple revenue: ${aapl.get_revenue():,.0f}") print(f"Microsoft revenue: ${msft.get_revenue():,.0f}")` **Why this API?** - Multiple companies - Standardized metrics ensure apples-to-apples comparison - Simple API for common metrics * * * ### Scenario 3: "Get Apple's revenue by product segment" **Recommended: XBRL API** `from edgar import Company filing = Company("AAPL").get_filings(form="10-K").latest() xbrl = filing.xbrl() # Find revenue statement with segments revenue_stmt = xbrl.statements.get("Revenues") print(revenue_stmt) # Shows dimensional breakdown` **Why this API?** - Need dimensional/segment data - Company Facts and Financials don't include segments - Full access to structured XBRL data * * * ### Scenario 4: "Get footnote details about debt terms" **Recommended: XBRL API** `from edgar import Company filing = Company("AAPL").get_filings(form="10-K").latest() xbrl = filing.xbrl() # Access footnotes for fact in xbrl.facts: if "Debt" in fact.concept and fact.footnote: print(f"{fact.concept}: {fact.footnote}")` **Why this API?** - Only XBRL API provides footnote access - Need detailed qualitative information - Going beyond just numbers * * * The Same Task, Three Ways ------------------------- Here's how to get current year revenue using each API: ### Method 1: Company Facts `company = Company("AAPL") income = company.income_statement() revenue = income.get_value("Revenues", period="latest") print(f"Revenue: ${revenue:,.0f}")` **Pros**: Simplest, one company object **Cons**: Less standardized concept names ### Method 2: Financials `company = Company("AAPL") financials = company.get_financials() revenue = financials.get_revenue() print(f"Revenue: ${revenue:,.0f}")` **Pros**: Standardized, guaranteed to work across companies **Cons**: Two API calls ### Method 3: XBRL `filing = Company("AAPL").get_filings(form="10-K").latest() statements = filing.xbrl().statements income = statements.income_statement revenue = income.get_fact_value("Revenues", period_filter="current") print(f"Revenue: ${revenue:,.0f}")` **Pros**: Most control, access to everything **Cons**: Most verbose, must filter period * * * When to Upgrade Your Approach ----------------------------- Start simple and upgrade only when you need more power: ### Start: Company Facts API Begin here for exploratory analysis and single-company work. `company = Company("AAPL") income = company.income_statement()` ### Upgrade to: Financials API When you need: - Cross-company comparison - Standardized metric names - Guaranteed concept availability `companies = [Company(ticker).get_financials() for ticker in ["AAPL", "MSFT", "GOOGL"]]` ### Upgrade to: XBRL API When you need: - Segment/dimension data - Footnotes and context - Custom or rare concepts - Maximum control over data `xbrl = filing.xbrl() statements = xbrl.statements` * * * Common Mistakes --------------- ### Mistake 1: Using XBRL for simple tasks `# DON'T: Too complex for this task filing = Company("AAPL").get_filings(form="10-K").latest() xbrl = filing.xbrl() statements = xbrl.statements income = statements.income_statement revenue = income.get_fact_value("Revenues") # DO: Use Company Facts company = Company("AAPL") income = company.income_statement() revenue = income.get_value("Revenues")` ### Mistake 2: Using Company Facts for cross-company work `# DON'T: Manual standardization aapl_income = Company("AAPL").income_statement() msft_income = Company("MSFT").income_statement() # Now you have to handle different concept names... # DO: Use Financials API aapl_fin = Company("AAPL").get_financials() msft_fin = Company("MSFT").get_financials() # Standardized getters work across companies` ### Mistake 3: Expecting segments in Company Facts `# DON'T: Company Facts doesn't have segments income = company.income_statement() segments = income.get_segments() # Won't work # DO: Use XBRL API for segments xbrl = filing.xbrl() # Access dimensional data through XBRL` * * * Summary ------- **Use Company Facts API when:** - Getting historical data for one company - Working with standard statements (income, balance, cash flow) - Speed matters - You want the simplest code **Use Financials API when:** - Comparing multiple companies - Need standardized metrics - Building cross-company datasets - Want guaranteed concept availability **Use XBRL API when:** - Need segment/dimension data - Accessing footnotes - Working with custom concepts - Building specialized tools - Maximum control is required **General Rule**: Start with the simplest API that meets your needs. You can always upgrade later. Back to top --- # MCP Server & Skills - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/ai-integration/#ai-integration) AI Integration ============== EdgarTools provides two AI integration features: 1. **MCP Server** -- Gives any MCP-compatible AI client direct access to SEC filing data through specialized tools 2. **Skills** -- Teaches Claude how to write better EdgarTools code by providing structured patterns and best practices Both are optional. Install with: `pip install "edgartools[ai]"` MCP Server ---------- The [Model Context Protocol](https://modelcontextprotocol.io/) server gives any MCP-compatible AI client access to SEC filing data -- whether that's a developer's Claude Desktop, a team's shared server, or a containerized deployment. No code required. ### Setup Choose the deployment method that fits your use case: #### uvx (recommended -- zero install) Ideal for individual use or scripted deployment. Requires [uv](https://docs.astral.sh/uv/getting-started/installation/) . `{ "mcpServers": { "edgartools": { "command": "uvx", "args": ["--from", "edgartools[ai]", "edgartools-mcp"], "env": { "EDGAR_IDENTITY": "Your Name your.email@example.com" } } } }` If you get a "spawn uvx ENOENT" error on macOS, use the full path to uvx (find it with `which uvx`). #### Python When edgartools is already installed in your environment: `{ "mcpServers": { "edgartools": { "command": "python3", "args": ["-m", "edgar.ai"], "env": { "EDGAR_IDENTITY": "Your Name your.email@example.com" } } } }` On Windows, use `python` instead of `python3`. #### Docker For server or production deployments where you want isolation and reproducibility: `FROM python:3.12-slim RUN pip install "edgartools[ai]" ENV EDGAR_IDENTITY="Your Name your.email@example.com" ENTRYPOINT ["python", "-m", "edgar.ai"]` Build and run: `docker build -t edgartools-mcp . docker run -i edgartools-mcp` The community also maintains Docker images -- see [hackerdogs/edgartools-mcp](https://hub.docker.com/r/hackerdogs/edgartools-mcp) on Docker Hub for a ready-to-use container with config templates for multiple MCP clients. #### HTTP transport (remote / team deployment) For shared servers, remote deployment, or registry-listed instances, run with Streamable HTTP transport instead of stdio: `edgartools-mcp --transport streamable-http --port 8000` This starts the server on `http://0.0.0.0:8000/mcp`. Clients connect with a URL instead of launching a subprocess: `{ "mcpServers": { "edgartools": { "url": "http://your-server:8000/mcp" } } }` CLI flags: | Flag | Default | Description | | --- | --- | --- | | `--transport` | `stdio` | `stdio` or `streamable-http` | | `--host` | `0.0.0.0` | Bind address for HTTP server | | `--port` | `8000` | Listen port for HTTP server | Docker with HTTP transport: `FROM python:3.12-slim RUN pip install "edgartools[ai]" ENV EDGAR_IDENTITY="Your Name your.email@example.com" ENTRYPOINT ["edgartools-mcp", "--transport", "streamable-http"] EXPOSE 8000` Replace `Your Name your.email@example.com` with your actual name and email. The SEC requires this to identify API users. **Verify** `python -m edgar.ai --test` **For Claude Desktop**, add the config above to your config file (`~/Library/Application Support/Claude/claude_desktop_config.json` on macOS, `%APPDATA%\Claude\claude_desktop_config.json` on Windows) and restart. You should see the MCP tools icon in the chat input. > **[edgar.tools also runs a hosted MCP server with AI-enriched data — no local setup needed →](https://app.edgar.tools/docs/mcp/setup?utm_source=edgartools-docs&utm_medium=see-live&utm_content=ai-integration) > ** ### Available Tools #### edgar\_company Get company profile, financials, recent filings, and ownership in one call. | Parameter | Description | | --- | --- | | `identifier` | Ticker, CIK, or company name (required) | | `include` | Sections to return: `profile`, `financials`, `filings`, `ownership` | | `periods` | Number of financial periods (default: 4) | | `annual` | Annual vs quarterly data (default: true) | **Try asking Claude:** * "Show me Apple's profile and latest financials" * "Get Microsoft's recent filings and ownership data" #### edgar\_search Search for companies or filings. | Parameter | Description | | --- | --- | | `query` | Search keywords (required) | | `search_type` | `companies`, `filings`, or `all` | | `identifier` | Limit to a specific company | | `form` | Filter by form type (e.g., `10-K`, `8-K`) | | `limit` | Max results (default: 10) | **Try asking Claude:** * "Search for semiconductor companies" * "Find Apple's 10-K filings" #### edgar\_filing Read filing content or specific sections. | Parameter | Description | | --- | --- | | `accession_number` | SEC accession number | | `identifier` + `form` | Alternative: company + form type | | `sections` | `summary`, `business`, `risk_factors`, `mda`, `financials`, or `all` | **Try asking Claude:** * "Show me the risk factors from Apple's latest 10-K" * "Get the MD&A section from Tesla's most recent annual report" #### edgar\_compare Compare companies side-by-side or analyze an industry. | Parameter | Description | | --- | --- | | `identifiers` | List of tickers/CIKs to compare | | `industry` | Alternative: industry name | | `metrics` | Metrics to compare (e.g., `revenue`, `net_income`) | | `periods` | Number of periods (default: 4) | **Try asking Claude:** * "Compare Apple, Microsoft, and Google on revenue and net income" * "How do the top semiconductor companies compare?" #### edgar\_ownership Insider transactions or fund portfolios. | Parameter | Description | | --- | --- | | `identifier` | Ticker, CIK, or fund CIK (required) | | `analysis_type` | `insiders`, `fund_portfolio`, or `portfolio_diff` | | `limit` | Max results (default: 20) | **Try asking Claude:** * "Show me recent insider transactions at Apple" * "What stocks does Berkshire Hathaway hold?" #### edgar\_monitor Get the latest SEC filings in real-time. | Parameter | Description | | --- | --- | | `form` | Filter by form type (e.g., `8-K`, `4`) | | `limit` | Max results (default: 20) | **Try asking Claude:** * "What SEC filings were just submitted?" * "Show me recent 8-K filings" #### edgar\_trends Get financial time series with growth rates. | Parameter | Description | | --- | --- | | `identifier` | Ticker, CIK, or company name (required) | | `concepts` | Metrics to track (e.g., `revenue`, `net_income`, `eps`) | | `periods` | Number of periods (default: 5) | **Try asking Claude:** * "Show me Apple's revenue trend over 5 years" * "What is Microsoft's EPS growth trajectory?" #### edgar\_screen Discover companies by industry, exchange, or state. | Parameter | Description | | --- | --- | | `industry` | Industry keyword | | `exchange` | Exchange name (e.g., `NYSE`, `Nasdaq`) | | `state` | State of incorporation (2-letter code) | | `limit` | Max results (default: 20) | **Try asking Claude:** * "Find pharmaceutical companies on NYSE" * "What software companies are in Delaware?" #### edgar\_text\_search Full-text search across SEC filing content. | Parameter | Description | | --- | --- | | `query` | Search text (required) | | `identifier` | Limit to a specific company | | `forms` | Filter by form types (e.g., `["8-K", "10-K"]`) | | `start_date` | Start date filter | **Try asking Claude:** * "Search for filings mentioning artificial intelligence" * "Find 8-K filings about cybersecurity incidents" #### edgar\_fund Get fund, ETF, BDC, and money market fund data. | Parameter | Description | | --- | --- | | `action` | `lookup`, `search`, `portfolio`, `money_market`, `bdc_search`, or `bdc_portfolio` (required) | | `identifier` | Fund ticker, series ID, or CIK | | `query` | Search text for fund or BDC name | | `limit` | Max results (default: 20) | **Try asking Claude:** * "Look up the Vanguard 500 Index Fund" * "Show me SPY's portfolio holdings" #### edgar\_proxy Get executive compensation and governance data from DEF 14A proxy statements. | Parameter | Description | | --- | --- | | `identifier` | Ticker, CIK, or company name (required) | | `filing_index` | Which proxy filing, 0=latest (default: 0) | **Try asking Claude:** * "What is Apple's CEO compensation?" * "Show me Microsoft's pay vs performance data" ### Any MCP Client The server works with any MCP-compatible client -- Claude Desktop, Cline, Continue.dev, or your own tooling. The configuration is the same `mcpServers` block regardless of client: `{ "mcpServers": { "edgartools": { "command": "uvx", "args": ["--from", "edgartools[ai]", "edgartools-mcp"], "env": { "EDGAR_IDENTITY": "Your Name your.email@example.com" } } } }` Where it goes depends on the client: Claude Desktop config file, Cline MCP settings, `~/.continue/config.json`, etc. edgar.tools also runs a hosted MCP server The local edgartools MCP server queries EDGAR directly through Python. The **[edgar.tools hosted MCP server](https://app.edgar.tools/docs/mcp/setup?utm_source=edgartools-docs&utm_medium=see-live&utm_content=ai-integration) ** adds AI-enriched data processed server-side: | Capability | Local (edgartools) | Hosted (edgar.tools) | | --- | --- | --- | | Material events | Basic 8-K parsing | LLM-classified event types | | Disclosure search | — | 12 XBRL topic clusters, all years | | Insider data | Individual Form 4s | 802K+ transactions with sentiment | | Filing sections | Raw text | AI summaries and key takeaways | Free tier: truncated MCP responses. Professional ($24.99/mo): full results. **[Set up the hosted MCP server →](https://app.edgar.tools/docs/mcp/setup?utm_source=edgartools-docs&utm_medium=see-live&utm_content=ai-integration) ** Skills ------ Skills are structured documentation packages that teach Claude how to write better EdgarTools code. They guide Claude to use the right APIs, avoid common mistakes, and follow best practices. ### What Do Skills Do? Without skills, Claude might write verbose code using low-level APIs: `# Without skills -- verbose, fragile facts = company.get_facts() income = facts.income_statement(periods=1, annual=True) if income is not None and not income.empty: if 'Revenue' in income.columns: revenue = income['Revenue'].iloc[0]` With skills, Claude writes idiomatic code: `# With skills -- clean, correct financials = company.get_financials() revenue = financials.get_revenue()` Skills cover patterns, sharp edges (common mistakes), and API routing decisions across six domains. ### Installing Skills **For Claude Code** (auto-discovered): `from edgar.ai import install_skill install_skill() # Installs to ~/.claude/skills/edgartools/` **For Claude Desktop** (upload as project knowledge): `from edgar.ai import package_skill package_skill() # Creates edgartools.zip` Upload the ZIP to a Claude Desktop Project. ### Skill Domains | Domain | What It Covers | | --- | --- | | **core** | Company lookup, filing search, API routing, quick reference | | **financials** | Financial statements, metrics, multi-company comparison | | **holdings** | 13F filings, institutional portfolios | | **ownership** | Insider transactions (Form 4), ownership summaries | | **reports** | 10-K, 10-Q, 8-K document sections | | **xbrl** | XBRL fact extraction, statement rendering | ### When to Use Which | I want to... | Use | | --- | --- | | Ask Claude questions about companies/filings | MCP Server | | Have Claude write EdgarTools code for me | Skills | | Both | Install both -- they complement each other | Built-in AI Features -------------------- These work without the `[ai]` extra. ### .docs Property Every major EdgarTools object has a `.docs` property with searchable API documentation: `from edgar import Company company = Company("AAPL") company.docs # Full API reference company.docs.search("financials") # Search for specific topics` Available on: `Company`, `Filing`, `Filings`, `XBRL`, `Statement` ### .to\_context() Method Token-efficient output optimized for LLM context windows: `company = Company("AAPL") # Control detail level company.to_context(detail='minimal') # ~100 tokens company.to_context(detail='standard') # ~300 tokens (default) company.to_context(detail='full') # ~500 tokens # Hard token limit company.to_context(max_tokens=200)` Available on: `Company`, `Filing`, `Filings`, `XBRL`, `Statement`, and most data objects. Troubleshooting --------------- **"EDGAR\_IDENTITY environment variable is required"** Add your name and email to the `env` section of your MCP config. The SEC requires identification for API access. **"Module edgar.ai not found"** Install with AI extras: `pip install "edgartools[ai]"` **"python3: command not found" (Windows)** Use `python` instead of `python3` in your MCP config. **MCP server not appearing in Claude Desktop** 1. Check the config file location is correct for your OS 2. Validate JSON syntax 3. Restart Claude Desktop completely (quit and relaunch) 4. Run `python -m edgar.ai --test` to verify **Skills not being picked up** 1. Verify installation: `ls ~/.claude/skills/edgartools/` 2. For Claude Desktop, upload as ZIP to a Project instead 3. Skills only affect code generation, not conversational responses Back to top --- # Understanding SEC Filings - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/concepts/sec-filings/#understanding-sec-filings) Understanding SEC Filings ========================= Introduction ------------ The U.S. Securities and Exchange Commission (SEC) requires public companies, investment funds, and certain individuals to submit various regulatory filings. These documents provide transparency into financial performance, significant events, insider activities, and investment decisions. The SEC's Electronic Data Gathering, Analysis, and Retrieval system (EDGAR) makes these filings publicly available. This guide explains the key SEC filing types, their purposes, and how to access and analyze them using the `edgartools` library. Why SEC Filings Matter ---------------------- SEC filings are the most authoritative source of company information available to the public. Unlike press releases, investor presentations, or news articles, SEC filings: * Are legally required to be accurate and complete * Follow standardized formats for consistency * Contain detailed financial data and disclosures * Are subject to strict liability for false or misleading information * Provide a historical record of a company's development Common SEC Filing Types ----------------------- ### Company Reporting Forms | Form | Description | Frequency | Key Information | | --- | --- | --- | --- | | **10-K** | Annual report | Annual | Comprehensive financial statements, business description, risk factors, management discussion | | **10-Q** | Quarterly report | Quarterly | Interim financial statements, updates since last 10-K | | **8-K** | Current report | As needed | Material events (acquisitions, executive changes, bankruptcy) | | **S-1** | Registration statement | Before IPO | Business model, financials, risk factors, use of proceeds | | **DEF 14A** | Proxy statement | Annual | Executive compensation, board members, shareholder proposals | ### Ownership and Investment Forms | Form | Description | Filed By | Key Information | | --- | --- | --- | --- | | **Form 3** | Initial ownership | Insiders | Initial positions when becoming an insider | | **Form 4** | Changes in ownership | Insiders | Purchases, sales, and other transactions | | **Form 5** | Annual ownership | Insiders | Summary of transactions for the year | | **13F** | Holdings report | Investment funds | Portfolio holdings of investment managers | | **13D/G** | Beneficial ownership | 5%+ shareholders | Significant ownership positions and intentions | Anatomy of Key Filings ---------------------- ### 10-K Annual Report The 10-K is the most comprehensive filing and typically contains: 1. **Business Overview** (Part I, Item 1) 2. Company operations, products/services, markets 3. Revenue breakdown by segment 4. Competitive landscape 5. **Risk Factors** (Part I, Item 1A) 6. Detailed disclosure of business risks 7. Industry, operational, and financial risks 8. **Management's Discussion & Analysis** (Part II, Item 7) 9. Analysis of financial condition and results 10. Liquidity and capital resources 11. Critical accounting policies 12. **Financial Statements** (Part II, Item 8) 13. Balance sheet 14. Income statement 15. Cash flow statement 16. Statement of shareholders' equity 17. Notes to financial statements 18. **Controls and Procedures** (Part II, Item 9) 19. Disclosure controls 20. Internal control over financial reporting ### 10-Q Quarterly Report The 10-Q is a condensed version of the 10-K filed quarterly, containing: * Unaudited financial statements * Management's discussion of results * Updates on risk factors * Disclosure of material events ### 8-K Current Report The 8-K reports significant events that occur between 10-K and 10-Q filings: * Item 1.01: Entry into a Material Agreement * Item 2.01: Completion of Acquisition or Disposition * Item 5.02: Departure/Election of Directors or Officers * Item 7.01: Regulation FD Disclosure * Item 8.01: Other Events ### Form 4 (Insider Transactions) Form 4 discloses transactions by company insiders (directors, officers, 10%+ shareholders): * Transaction date and type (purchase, sale, grant, exercise) * Number of securities involved * Price per share * Resulting ownership after transaction ### 13F (Investment Fund Holdings) 13F reports show investment portfolios of funds managing over $100 million: * Securities held at quarter-end * Number of shares * Market value * Investment discretion Working with SEC Filings in edgartools -------------------------------------- ### Accessing Filings `from edgar import Company # Get all filings for a specific company apple = Company("AAPL") filings = apple.get_filings() # Filter by form type annual_reports = apple.get_filings(form="10-K") quarterly_reports = apple.get_filings(form="10-Q") current_reports = apple.get_filings(form="8-K") # Get the most recent annual report latest_10k = annual_reports.latest()` ### Extracting Financial Data The simplest way to get financial statements is through the `get_financials()` method on a Company: `# Get financial statements (recommended approach) company = Company("AAPL") financials = company.get_financials() # Access specific statements balance_sheet = financials.balance_sheet() income_stmt = financials.income_statement() cash_flow = financials.cashflow_statement() # Get specific values directly revenue = financials.get_revenue() net_income = financials.get_net_income() total_assets = financials.get_total_assets()` ### Analyzing Insider Trading `from edgar import Company # Get Form 4 filings for a company tesla = Company("TSLA") form4_filings = tesla.get_filings(form=4) # Parse the most recent filing form4 = form4_filings.latest(1).obj() # Get the transaction summary summary = form4.get_ownership_summary() print(f"Insider: {summary.insider_name}") print(f"Position: {summary.position}") print(f"Activity: {summary.primary_activity}") print(f"Net shares changed: {summary.net_change}")` ### Researching Investment Fund Holdings `from edgar import get_filings # Get 13F filings (institutional holdings reports) thirteenf_filings = get_filings(form="13F-HR") # Parse a specific fund's holdings filing = thirteenf_filings[0] thirteenf = filing.obj() # View top holdings print(thirteenf.holdings)` Best Practices for Working with SEC Filings ------------------------------------------- ### 1\. Understand Filing Timelines * **10-K**: Due 60-90 days after fiscal year-end (depending on company size) * **10-Q**: Due 40-45 days after quarter-end * **8-K**: Due within 4 business days of the event * **Form 4**: Due within 2 business days of the transaction * **13F**: Due within 45 days of quarter-end ### 2\. Be Aware of Filing Amendments Amendments are indicated with a suffix: - 10-K/A, 10-Q/A, 8-K/A, etc. `# Get original and amended filings filings = company.get_filings(form="10-K") amendments = filings.filter(form="10-K/A")` ### 3\. Handle Historical Data Carefully * Financial restatements can change historical data * Company structures change over time (mergers, spin-offs) * Accounting standards evolve ### 4\. Respect SEC Access Guidelines The SEC has rate limits for EDGAR access: - Identify yourself properly with `edgar.set_identity()` - Implement appropriate delays between requests - Consider using local caching for repeated access `from edgar import set_identity # Set your identity for SEC access set_identity("Your Name your.email@example.com")` Conclusion ---------- SEC filings provide a wealth of structured and unstructured data for financial analysis, investment research, and regulatory compliance. With `edgartools`, you can efficiently access, parse, and analyze these filings to extract valuable insights. Understanding the different filing types, their purposes, and how to work with them programmatically allows you to build sophisticated financial analysis workflows and make more informed investment decisions. Additional Resources -------------------- * [SEC EDGAR Website](https://www.sec.gov/edgar/search-and-access) * [SEC Filing Deadlines](https://www.sec.gov/edgar/filer-information/calendar) * [EDGAR Filing Codes](https://www.sec.gov/info/edgar/forms/edgform.pdf) Back to top --- # Python SEC Filings Tutorials | Free Colab Notebooks | EdgarTools - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/notebooks/#sec-filings-python-tutorials-free-colab-notebooks) SEC Filings Python Tutorials — Free Colab Notebooks =================================================== Access SEC EDGAR data with Python — completely **free**, no API key or paid subscription required. Every notebook runs instantly in **Google Colab** with one click. Install locally with `pip install edgartools`. [**Get Started**](https://edgartools.readthedocs.io/en/latest/notebooks/#getting-started) | [View on GitHub](https://github.com/dgunning/edgartools/tree/main/notebooks) * * * Getting Started --------------- New to edgartools? Start here. These notebooks cover installation, company lookups, and your first SEC filing queries. | Notebook | Difficulty | Links | | --- | --- | --- | | **SEC EDGAR API in Python** — Comprehensive overview: companies, filings, financials, insiders, and holdings in one notebook | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-edgar-api-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/sec-edgar-api-python.ipynb) | | **SEC Company Data with Python** — Look up any company by ticker or CIK: metadata, filing history, industry classification | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-company-data-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/sec-company-data-python.ipynb) | | **Getting Started with SEC Filings** — First steps: install, configure, and pull your first filing | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/01_getting_started.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/01_getting_started.ipynb) | | **Beginner's Guide to EdgarTools** — Complete walkthrough of the core API | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Beginners-Guide.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/Beginners-Guide.ipynb) | | **Search SEC Filings by Ticker Symbol** — Find filings by ticker, CIK, or company name | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Ticker-Search-with-edgartools.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/Ticker-Search-with-edgartools.ipynb) | | **Troubleshooting SSL Issues** — Fix SSL/TLS connection problems in corporate or restricted environments | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/02_troubleshooting_ssl.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/02_troubleshooting_ssl.ipynb) | * * * Filings ------- Search, filter, download, and analyze SEC filings. From today's filings to bulk downloads to full 10-K/10-Q/8-K parsing. | Notebook | Difficulty | Links | | --- | --- | --- | | **Search and Filter SEC Filings** — Search across all companies by date, form type, or quarter | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/search-sec-filings-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/search-sec-filings-python.ipynb) | | **Get Today's SEC Filings** — Real-time access to filings submitted today | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-filings-today-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/sec-filings-today-python.ipynb) | | **Monitor Filings for Multiple Companies** — Watch multiple tickers for new SEC submissions | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/monitor-sec-filings-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/monitor-sec-filings-python.ipynb) | | **Download 10-K Annual Reports** — Download and parse 10-K annual reports | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/download-10k-annual-report-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/download-10k-annual-report-python.ipynb) | | **Analyze 10-K Annual Reports** — Extract business description, risk factors, MD&A, and financials | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/analyze-10k-annual-report-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/analyze-10k-annual-report-python.ipynb) | | **Extract Business Description from 10-K Item 1** — Pull the company overview section from annual reports | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/10k-business-description-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/10k-business-description-python.ipynb) | | **10-Q Quarterly Earnings** — Parse quarterly financial data from 10-Q filings | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/10q-quarterly-earnings-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/10q-quarterly-earnings-python.ipynb) | | **Extract 8-K Earnings Releases** — Pull earnings announcements from 8-K current event reports | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/8k-earnings-release-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/8k-earnings-release-python.ipynb) | | **Download SEC Filings in Bulk** — Batch download filings across companies and date ranges | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/download-sec-filings-bulk-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/download-sec-filings-bulk-python.ipynb) | | **Filter Companies by Industry and SIC Code** — Find companies by sector using SEC industry classifications | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-industry-sic-code-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/sec-industry-sic-code-python.ipynb) | | **SEC Filing Text for NLP** — Extract raw text from filings for natural language processing | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-filing-text-nlp-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/sec-filing-text-nlp-python.ipynb) | | **Filing Exhibits and Attachments** — Access exhibits, press releases, and supplemental documents | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-filing-exhibits-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/sec-filing-exhibits-python.ipynb) | | **Analyze SEC Comment Letters** — Parse CORRESP filings: SEC staff questions and company responses | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-comment-letters-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/sec-comment-letters-python.ipynb) | | **Browse and Page Through Filings** — Navigate large filing collections with paging | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Paging-Through-Filings.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/Paging-Through-Filings.ipynb) | | **Working with Filing Attachments** — Access individual documents within a filing | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Beginners-filings-attachments.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/Beginners-filings-attachments.ipynb) | * * * Financial Statements -------------------- Extract income statements, balance sheets, and cash flow statements from SEC filings. Compare financials across companies and time periods. | Notebook | Difficulty | Links | | --- | --- | --- | | **Financial Statements from SEC Filings** — Extract income statements, balance sheets, and cash flows from 10-K/10-Q | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/financial-statements-sec-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/financial-statements-sec-python.ipynb) | | **Extract Revenue and Earnings** — Pull revenue, net income, and EPS from SEC filings | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/extract-revenue-earnings-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/extract-revenue-earnings-python.ipynb) | | **Compare Company Financials** — Side-by-side financial comparisons across companies | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/compare-company-financials-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/compare-company-financials-python.ipynb) | | **Viewing Financial Statements** — Display and navigate financial statement tables | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Viewing-Financial-Statements.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/Viewing-Financial-Statements.ipynb) | | **Financial Statements to DataFrame** — Build quarterly IS, BS, and CF DataFrames with multi-index (Ticker, Period) for multiple companies | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/financial-statements-to-dataframe.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/financial-statements-to-dataframe.ipynb) | Prefer a visual interface? Every tutorial above runs as Python code. **[edgar.tools](https://app.edgar.tools/?utm_source=edgartools-docs&utm_medium=see-live&utm_content=notebooks) ** puts the same data in a web UI — no code, no notebooks, no setup. * **[Browse any company's filings and financials →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=notebooks) ** * **[See filings arrive in real time →](https://app.edgar.tools/filings?utm_source=edgartools-docs&utm_medium=see-live&utm_content=notebooks) ** * **[Explore insider trades with sentiment analysis →](https://app.edgar.tools/companies/TSLA?utm_source=edgartools-docs&utm_medium=see-live&utm_content=notebooks) ** Free tier available. Also includes a REST API and hosted MCP server for AI integrations. * * * XBRL Deep Dive -------------- Parse structured XBRL financial data — fact queries, multi-period views, custom tags, and advanced statement analysis. | Notebook | Difficulty | Links | | --- | --- | --- | | **Parse XBRL Financial Data** — Work with XBRL-tagged data: income statements, balance sheets, disclosures | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/xbrl-financial-data-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/xbrl-financial-data-python.ipynb) | | **Read Data from XBRL** — Extract structured data from XBRL documents | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Reading-Data-From-XBRL.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/Reading-Data-From-XBRL.ipynb) | | **XBRL Fact Queries** — Query individual XBRL facts with the enhanced API | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/XBRL2-FactQueries.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/XBRL2-FactQueries.ipynb) | | **Multi-Period Financial Views** — Compare financial statements across quarters and years | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/XBRL2-PeriodViews.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/XBRL2-PeriodViews.ipynb) | | **Cash Flow Statements** — Analyze cash flow statements in detail | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/XBRL2-Cashflow-Statements.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/XBRL2-Cashflow-Statements.ipynb) | | **Standardized Financial Statements** — Map company-specific tags to standardized concepts | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/XBRL2-StandardizedStatements.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/XBRL2-StandardizedStatements.ipynb) | | **Quarterly Financial Statements** — Quarterly statement analysis and comparison | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/XBRL2-QuarterlyStatements.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/XBRL2-QuarterlyStatements.ipynb) | | **Stitch Statements Across Filings** — Combine statements from multiple filings into time series | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/XBRL2-StitchingStatements.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/XBRL2-StitchingStatements.ipynb) | | **Custom XBRL Tags** — Handle company-specific extension tags | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/XBRL2-CustomTags.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/XBRL2-CustomTags.ipynb) | | **Non-Financial Statements** — Segment disclosures and non-financial XBRL data | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/XBRL2-NonFinancialStatements.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/XBRL2-NonFinancialStatements.ipynb) | | **Instance-Only XBRL** — Parse XBRL documents without a taxonomy schema | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/XBRL2-Instance-Only-XBRL.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/XBRL2-Instance-Only-XBRL.ipynb) | | **Explore XBRL Concepts** — Browse the XBRL taxonomy and concept hierarchy | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/XBRLConcepts.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/XBRLConcepts.ipynb) | * * * Insider Trading --------------- Track insider buying, selling, and ownership changes from SEC Form 3 and Form 4 filings. | Notebook | Difficulty | Links | | --- | --- | --- | | **Track Insider Trading from Form 4** — Monitor officer and director buys, sells, and option exercises | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/insider-trading-sec-form4-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/insider-trading-sec-form4-python.ipynb) | | **Initial Insider Ownership (Form 3)** — Analyze initial ownership disclosures for new insiders | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Initial-Insider-Transactions.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/Initial-Insider-Transactions.ipynb) | * * * Beneficial Ownership -------------------- Track large shareholders and activist investors through SEC Schedule 13D/G filings. | Notebook | Difficulty | Links | | --- | --- | --- | | **Beneficial Ownership (Schedule 13D/G)** — Track 5%+ shareholders, activist positions, and ownership changes | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/beneficial-ownership-sec-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/beneficial-ownership-sec-python.ipynb) | * * * Institutional Holdings ---------------------- Analyze what hedge funds, mutual funds, and large investors are buying and selling. | Notebook | Difficulty | Links | | --- | --- | --- | | **13F Institutional Holdings** — Quarterly portfolio disclosures from hedge funds and institutional investors | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/13f-institutional-holdings-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/13f-institutional-holdings-python.ipynb) | * * * Investment Funds ---------------- Analyze mutual fund, ETF, and closed-end fund portfolios from SEC N-PORT and other fund filings. | Notebook | Difficulty | Links | | --- | --- | --- | | **Mutual Fund Holdings (N-PORT)** — Complete portfolio holdings from monthly N-PORT filings | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/mutual-fund-holdings-nport-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/mutual-fund-holdings-nport-python.ipynb) | | **Money Market Fund Holdings (N-MFP)** — Portfolio holdings, yields, NAV, and liquidity from monthly N-MFP filings | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/money-market-fund-nmfp-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/money-market-fund-nmfp-python.ipynb) | | **ETF and Fund Holdings** — Analyze ETF portfolios and fund composition | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/etf-fund-holdings-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/etf-fund-holdings-python.ipynb) | | **Fund Census (N-CEN)** — Annual fund census: series, service providers, directors, ETF mechanics, broker commissions | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/fund-census-ncen-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/fund-census-ncen-python.ipynb) | | **Fund Filing Types** — Overview of SEC fund filing types and structures | Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Fund-Filings.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/Fund-Filings.ipynb) | | **Fund Derivative Holdings** — Analyze derivative positions within fund portfolios | Advanced | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Fund-Derivatives.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/Fund-Derivatives.ipynb) | * * * Business Development Companies ------------------------------ Analyze BDC portfolio investments, lending activity, and SEC filings. | Notebook | Difficulty | Links | | --- | --- | --- | | **BDC SEC Filings and Portfolios** — Analyze BDC investment portfolios and lending activity | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/bdc-business-development-company-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/bdc-business-development-company-python.ipynb) | * * * Executive Compensation & Proxy Statements ----------------------------------------- Parse proxy statements for executive pay, board composition, and shareholder proposals. | Notebook | Difficulty | Links | | --- | --- | --- | | **Proxy Statements (DEF 14A)** — Parse proxy statements: proposals, board members, and voting items | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/proxy-statement-def14a-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/proxy-statement-def14a-python.ipynb) | | **Executive Compensation** — Extract CEO and executive pay from proxy statements | Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/executive-compensation-sec-python.ipynb)
  [GitHub](https://github.com/dgunning/edgartools/blob/main/notebooks/executive-compensation-sec-python.ipynb) | * * * Why EdgarTools? --------------- **sec-api.io** is a data access API -- it gives you JSON from REST endpoints, and you build the analysis yourself. **EdgarTools** is a data analysis library -- it parses filings into structured Python objects with built-in methods for the analysis you actually want to do. And it's free. ### The Core Difference With sec-api, getting 13F institutional holdings means calling an endpoint, receiving JSON, then writing code to compare quarters, calculate position changes, and format results. With edgartools, that analysis is built in: `from edgar import * # Parse a 13F filing into a structured object thirteenf = Company("BERKSHIRE HATHAWAY").get_filings(form="13F-HR")[0].obj() # Built-in quarter-over-quarter comparison thirteenf.compare_holdings() # NEW, CLOSED, INCREASED, DECREASED positions # Multi-quarter trend analysis with sparklines thirteenf.holding_history(periods=4) # All holdings as a pandas DataFrame, ready for analysis thirteenf.holdings_data()` sec-api returns the raw holdings data as JSON. The comparison logic, trend analysis, and DataFrame conversion are left to you. ### What You Get Out of the Box EdgarTools doesn't just fetch data -- it structures it into objects with properties, methods, and DataFrames designed for the analysis Python developers actually do: | Filing Type | What edgartools gives you | What a JSON API gives you | | --- | --- | --- | | **10-K / 10-Q** | `TenK` / `TenQ` objects with `.financials`, section extraction, multi-period statements | Raw XBRL JSON -- you build the statement structure | | **8-K** | `EightK` with item-level parsing, earnings extraction | Section text or structured fields for a few items | | **13F** | `ThirteenF` with `compare_holdings()`, `holding_history()`, sparklines | Holdings array -- you write the diff logic | | **N-PORT** | `FundReport` with `investment_data()`, asset allocation, country exposure | Holdings array -- you aggregate and categorize | | **N-MFP** | `MoneyMarketFund` with yield/NAV/liquidity time series, category breakdowns | \-- | | **N-CEN** | `FundCensus` with series, providers, broker commissions, board composition | \-- | | **DEF 14A** | `ProxyStatement` with executive compensation tables, board data | Separate exec comp and board endpoints | | **13D/G** | `Schedule13DG` with ownership parsing | Structured JSON | | **Form 4** | `Ownership` with transaction details | Structured JSON | ### Pricing | | EdgarTools | sec-api.io | | --- | --- | --- | | **Price** | Free forever | Free trial (100 calls), then $49-$239/mo | | **API key** | Not required | Required | | **Open source** | Yes (MIT license) | No | | **Works offline** | Yes (with local storage) | No | ### Use sec-api if you need... sec-api is a hosted platform backed by databases and infrastructure, so it can offer things a client-side library can't: real-time WebSocket filing streams, full-text boolean search across all filings, filing-to-PDF conversion, and prebuilt datasets like SEC enforcement actions and Form ADV investment adviser data. It also works with any programming language via REST, not just Python. ### Use EdgarTools if you need... EdgarTools is for Python developers who want to go straight from a filing to analysis. No API key, no HTTP plumbing, no JSON wrangling -- just `pip install edgartools` and you get structured objects with built-in analysis methods, pandas DataFrames, Rich terminal display, and native Jupyter/Colab support. Free forever, open source, and works offline. * * * Running on Google Colab ----------------------- Click any **Open in Colab** badge above, or: 1. Go to [colab.research.google.com](https://colab.research.google.com/) 2. **File > Open notebook > GitHub** tab 3. Enter `dgunning/edgartools` and select a notebook Resources --------- * [EdgarTools Documentation](https://edgartools.io/) * [GitHub Repository](https://github.com/dgunning/edgartools) * [PyPI Package](https://pypi.org/project/edgartools/) * [Installation Guide](https://edgartools.readthedocs.io/en/latest/installation/) * [Quick Start](https://edgartools.readthedocs.io/en/latest/quickstart/) Back to top --- # Common Pitfalls - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/common-pitfalls/#common-pitfalls) Common Pitfalls =============== New to EdgarTools? This page covers the most frequent mistakes and how to avoid them. * * * Which API Should I Use? ----------------------- EdgarTools has three ways to get financial data. This causes the most confusion for new users. | I want to... | Use this | Not this | | --- | --- | --- | | Get revenue, net income, balance sheet | `company.get_financials()` | `company.get_facts()` or `filing.xbrl()` | | Compare Apple vs Microsoft financials | `company.get_financials()` for each | Manual XBRL parsing | | Get 5+ years of historical trends | `company.get_facts()` | Iterating over 5 filings | | Get segment breakdowns or footnotes | `filing.xbrl()` | `get_financials()` (no segments) | **Rule of thumb:** Start with `get_financials()`. It covers 95% of use cases. Only use `get_facts()` for 4+ years of history, and `filing.xbrl()` for segments/footnotes. For a detailed comparison, see [Choosing the Right API](https://edgartools.readthedocs.io/en/stable/xbrl/getting-started/choosing-the-right-api/) . * * * Financial Data -------------- ### Use `get_financials()`, not `filing.xbrl()` `# WRONG: Too complex for simple financial data filing = Company("AAPL").get_filings(form="10-K").latest() xbrl = filing.xbrl() statements = xbrl.statements income = statements.income_statement() # RIGHT: Direct and simple financials = Company("AAPL").get_financials() income = financials.income_statement()` ### Use `get_financials()`, not `get_facts()` for standard data `# WRONG: get_facts() is for historical trends, not everyday use facts = company.get_facts() income = facts.income_statement(periods=3, annual=True) # RIGHT: get_financials() already includes 3 years financials = company.get_financials() income = financials.income_statement() # Already 3 years revenue = financials.get_revenue() # Single current value` ### Quick getters can return `None` Always check before doing math: `revenue = financials.get_revenue() net_income = financials.get_net_income() # WRONG: Will crash if either is None margin = net_income / revenue # RIGHT: Check first if revenue and net_income: margin = net_income / revenue` ### `period_offset=0` means current year, not prior year `this_year = financials.get_revenue() # period_offset=0 (default) last_year = financials.get_revenue(period_offset=1) # 1 year ago two_ago = financials.get_revenue(period_offset=2) # 2 years ago` ### Financial values are in actual dollars Values from `get_financials()` are in real dollars, not thousands or millions: `revenue = financials.get_revenue() # e.g. 391035000000 print(f"${revenue/1e9:.1f}B") # "$391.0B"` ### The cash flow method is `cashflow_statement()` The canonical method name is `cashflow_statement()` (no underscore between "cash" and "flow"). The alias `cash_flow_statement()` also works. All three statements follow the same pattern: `financials.income_statement() # Income statement financials.balance_sheet() # Balance sheet financials.cashflow_statement() # Cash flow statement (cash_flow_statement() also works)` * * * Filings ------- ### Always limit before iterating `# WRONG: Loads ALL filings into memory for filing in company.get_filings(): process(filing) # RIGHT: Limit first for filing in company.get_filings(form="10-K").head(10): process(filing.obj())` ### Use `.head(n)`, not `list(filings)[:n]` `# WRONG: Converts ALL filings to a list first recent = list(company.get_filings())[:5] # RIGHT: Only fetches 5 recent = company.get_filings().head(5)` ### Use `filing.obj()` for sections, not `filing.document()` `# WRONG: Raw document text, no structure doc = filing.document() text = doc.text() # RIGHT: Parsed data object with sections tenk = filing.obj() risk_factors = tenk['Item 1A'] business = tenk['Item 1']` * * * Insider Trading (Form 4) ------------------------ ### Use `get_ownership_summary()`, not raw DataFrames `summary = form4.get_ownership_summary() print(f"Activity: {summary.primary_activity}") # "Purchase", "Sale", "Mixed" print(f"Net shares: {summary.net_change:,}") # positive=buy, negative=sell print(f"Net value: ${summary.net_value:,.0f}")` ### `get_ownership_summary()` returns ONE object, not a list `# WRONG: Will crash — summary is not iterable summary = form4.get_ownership_summary() for item in summary: # TypeError! print(item.insider_name) # RIGHT: Access properties directly summary = form4.get_ownership_summary() summary.insider_name # "Tim Cook" summary.primary_activity # "Sale" # To process MULTIPLE filings, loop over filings: for filing in company.get_filings(form="4").head(20): summary = filing.obj().get_ownership_summary() print(f"{summary.insider_name}: {summary.primary_activity}")` ### Transaction codes have specific meanings | Code | Meaning | Signal | | --- | --- | --- | | P | Open market purchase | Strong buy signal | | S | Open market sale | Sell signal | | A | Grant/Award | Compensation, not a purchase | | M | Option exercise | Converting options, not buying | | F | Tax withholding | Shares sold to cover taxes | The `primary_activity` property on `TransactionSummary` interprets these for you. * * * Institutional Holdings (13F) ---------------------------- ### Holdings value is in $1,000s, not dollars `holdings = thirteenf.holdings # WRONG: Off by 1000x total = holdings['Value'].sum() # RIGHT: Multiply by 1000 total = holdings['Value'].sum() * 1000` ### Use CUSIP for reliable lookups, not ticker `# Ticker may be missing for some securities apple = holdings[holdings['Cusip'] == '037833100'] # Reliable apple = holdings[holdings['Ticker'] == 'AAPL'] # May miss some` ### Don't iterate all 13Fs to find holders of a stock `# WRONG: Iterating 8,000+ filings to find "who holds Apple?" for filing in get_filings(form="13F-HR"): holdings = filing.obj().holdings if 'AAPL' in holdings['Ticker'].values: print(filing.company) # RIGHT: Filter by filer first filing = get_filings(form="13F-HR").filter(company="BERKSHIRE")[0] holdings = filing.obj().holdings` ### 13F filings have a 45-day lag Holdings are reported as of quarter-end, but filings aren't due for 45 days. Q3 (Sep 30) data may not appear until mid-November. * * * 10-K / 10-Q / 8-K Reports ------------------------- ### Item keys differ between report types `# 10-K / 10-Q: "Item 1", "Item 1A", "Item 7" tenk['Item 1A'] # Risk Factors # 8-K: "Item 1.01", "Item 5.02" (with dots) eightk['Item 5.02'] # Personnel Changes` ### Not all items exist in every filing `risk_factors = tenk.risk_factors if risk_factors: print(risk_factors.text[:1000]) else: print("No risk factors section in this filing")` * * * Exporting Data -------------- ### Export statements, not the financials object `financials = company.get_financials() # WRONG: financials object has no to_dataframe() df = financials.to_dataframe() # AttributeError # RIGHT: call to_dataframe() on the individual statement df = financials.income_statement().to_dataframe() df = financials.balance_sheet().to_dataframe() df = financials.cashflow_statement().to_dataframe()` ### Export to CSV or Excel `income = company.get_financials().income_statement() # CSV income.to_dataframe().to_csv("income.csv") # Excel income.to_dataframe().to_excel("income.xlsx")` ### Export filings list (not financial data) `# Export a list of filings to a DataFrame filings = company.get_filings(form="10-K") df = filings.to_pandas() # One row per filing df.to_csv("10k_filings.csv")` ### Financial values are raw numbers (not formatted) `get_revenue()` returns an integer like `391035000000`, not `"$391B"`. Format it yourself: `revenue = financials.get_revenue() print(f"${revenue / 1e9:.1f}B") # "$391.0B"` * * * XBRL ---- ### `filing.xbrl()` can return `None` Not all filings have XBRL data. Always check: `xbrl = filing.xbrl() if xbrl: print(xbrl.entity_name) else: print("No XBRL data in this filing")` ### XBRL concept names vary by company Different companies use different concept names for the same thing (e.g., "Revenues" vs "RevenueFromContractWithCustomerExcludingAssessedTax"). Use `get_financials()` instead, which normalizes these automatically. `# Instead of guessing concept names: revenue = company.get_financials().get_revenue()` Back to top --- # Choosing the Right API - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/xbrl/getting-started/choosing-the-right-api/#choosing-the-right-api) Choosing the Right API ====================== EdgarTools offers three different ways to access financial data. This guide helps you choose the right one for your needs. Quick Decision Tree ------------------- `What do you want to do? ├─ "Get historical financial trends for one company" │ └─> Use Company Facts API (company.income_statement()) │ ├─ "Compare metrics across multiple companies" │ └─> Use Financials API (company.get_financials()) │ ├─ "Need segment data, dimensions, or detailed breakdowns" │ └─> Use XBRL API (filing.xbrl().statements) │ └─ "Need footnotes or custom concepts" └─> Use XBRL API (filing.xbrl())` The Three APIs at a Glance -------------------------- ### 1\. Company Facts API - Simplest `company = Company("AAPL") income = company.income_statement() # Multi-year data instantly` ### 2\. Financials API - Best for Comparison `company = Company("AAPL") financials = company.get_financials() revenue = financials.get_revenue()` ### 3\. XBRL API - Most Complete `filing = Company("AAPL").get_filings(form="10-K").latest() xbrl = filing.xbrl() statements = xbrl.statements` Detailed Comparison ------------------- | Feature | Company Facts | Financials | XBRL | | --- | --- | --- | --- | | **Speed** | Fastest (cached) | Fast | Slower (parses filing) | | **Lines of code** | 1-2 | 2-3 | 3-5 | | **Multi-period data** | Built-in | Built-in | Manual filtering | | **Historical range** | All available periods | Recent filings | Single filing only | | **Statements** | Primary 3 only | Primary 3 only | All statements | | **Segment/dimension data** | No | No | Yes | | **Footnotes** | No | No | Yes | | **Custom concepts** | No | Limited | All concepts | | **Standardization** | Partial | Yes | Raw (you control) | | **Cross-company comparison** | Manual | Built-in | Manual | Use Case Examples ----------------- ### Scenario 1: "I want Apple's revenue for the last 5 years" **Recommended: Company Facts API** `from edgar import Company company = Company("AAPL") income = company.income_statement() # Get all revenue values revenues = income.get_all_values("Revenues") for value in revenues[:5]: print(f"{value.period}: ${value.value:,.0f}")` **Why this API?** - Single company, historical trend - Standard metric (revenue) - Fastest way to get multi-period data * * * ### Scenario 2: "Compare revenue growth: Apple vs Microsoft" **Recommended: Financials API** `from edgar import Company aapl = Company("AAPL").get_financials() msft = Company("MSFT").get_financials() print(f"Apple revenue: ${aapl.get_revenue():,.0f}") print(f"Microsoft revenue: ${msft.get_revenue():,.0f}")` **Why this API?** - Multiple companies - Standardized metrics ensure apples-to-apples comparison - Simple API for common metrics * * * ### Scenario 3: "Get Apple's revenue by product segment" **Recommended: XBRL API** `from edgar import Company filing = Company("AAPL").get_filings(form="10-K").latest() xbrl = filing.xbrl() # Find revenue statement with segments revenue_stmt = xbrl.statements.get("Revenues") print(revenue_stmt) # Shows dimensional breakdown` **Why this API?** - Need dimensional/segment data - Company Facts and Financials don't include segments - Full access to structured XBRL data * * * ### Scenario 4: "Get footnote details about debt terms" **Recommended: XBRL API** `from edgar import Company filing = Company("AAPL").get_filings(form="10-K").latest() xbrl = filing.xbrl() # Access footnotes for fact in xbrl.facts: if "Debt" in fact.concept and fact.footnote: print(f"{fact.concept}: {fact.footnote}")` **Why this API?** - Only XBRL API provides footnote access - Need detailed qualitative information - Going beyond just numbers * * * The Same Task, Three Ways ------------------------- Here's how to get current year revenue using each API: ### Method 1: Company Facts `company = Company("AAPL") income = company.income_statement() revenue = income.get_value("Revenues", period="latest") print(f"Revenue: ${revenue:,.0f}")` **Pros**: Simplest, one company object **Cons**: Less standardized concept names ### Method 2: Financials `company = Company("AAPL") financials = company.get_financials() revenue = financials.get_revenue() print(f"Revenue: ${revenue:,.0f}")` **Pros**: Standardized, guaranteed to work across companies **Cons**: Two API calls ### Method 3: XBRL `filing = Company("AAPL").get_filings(form="10-K").latest() statements = filing.xbrl().statements income = statements.income_statement revenue = income.get_fact_value("Revenues", period_filter="current") print(f"Revenue: ${revenue:,.0f}")` **Pros**: Most control, access to everything **Cons**: Most verbose, must filter period * * * When to Upgrade Your Approach ----------------------------- Start simple and upgrade only when you need more power: ### Start: Company Facts API Begin here for exploratory analysis and single-company work. `company = Company("AAPL") income = company.income_statement()` ### Upgrade to: Financials API When you need: - Cross-company comparison - Standardized metric names - Guaranteed concept availability `companies = [Company(ticker).get_financials() for ticker in ["AAPL", "MSFT", "GOOGL"]]` ### Upgrade to: XBRL API When you need: - Segment/dimension data - Footnotes and context - Custom or rare concepts - Maximum control over data `xbrl = filing.xbrl() statements = xbrl.statements` * * * Common Mistakes --------------- ### Mistake 1: Using XBRL for simple tasks `# DON'T: Too complex for this task filing = Company("AAPL").get_filings(form="10-K").latest() xbrl = filing.xbrl() statements = xbrl.statements income = statements.income_statement revenue = income.get_fact_value("Revenues") # DO: Use Company Facts company = Company("AAPL") income = company.income_statement() revenue = income.get_value("Revenues")` ### Mistake 2: Using Company Facts for cross-company work `# DON'T: Manual standardization aapl_income = Company("AAPL").income_statement() msft_income = Company("MSFT").income_statement() # Now you have to handle different concept names... # DO: Use Financials API aapl_fin = Company("AAPL").get_financials() msft_fin = Company("MSFT").get_financials() # Standardized getters work across companies` ### Mistake 3: Expecting segments in Company Facts `# DON'T: Company Facts doesn't have segments income = company.income_statement() segments = income.get_segments() # Won't work # DO: Use XBRL API for segments xbrl = filing.xbrl() # Access dimensional data through XBRL` * * * Summary ------- **Use Company Facts API when:** - Getting historical data for one company - Working with standard statements (income, balance, cash flow) - Speed matters - You want the simplest code **Use Financials API when:** - Comparing multiple companies - Need standardized metrics - Building cross-company datasets - Want guaranteed concept availability **Use XBRL API when:** - Need segment/dimension data - Accessing footnotes - Working with custom concepts - Building specialized tools - Maximum control is required **General Rule**: Start with the simplest API that meets your needs. You can always upgrade later. Back to top --- # Quick Start - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/quickstart/#quick-start-guide) Quick Start Guide ================= Get up and running with EdgarTools in 5 minutes. By the end, you'll have a company's financial statements in Python. Prerequisites ------------- * Python 3.8 or higher * Internet connection * Basic familiarity with Python Step 1: Install EdgarTools -------------------------- `pip install edgartools` Trouble importing? If you see `ImportError: cannot import name 'get_filings' from 'edgar'`, you may have installed the wrong package. There is an unrelated package called `edgar` on PyPI. Fix it with: `pip uninstall edgar && pip install edgartools` Step 2: Set Your Identity ------------------------- The SEC requires all API users to identify themselves. Set your identity once: `from edgar import set_identity # Use your name and email (required by SEC) set_identity("John Doe john.doe@company.com")` **Tip:** You can also set the `EDGAR_IDENTITY` environment variable to avoid doing this in every script. Step 3: Get a Company --------------------- Look up any public company by ticker symbol or CIK number: `from edgar import Company company = Company("AAPL") # Apple Inc.` ![AAPL](https://edgartools.readthedocs.io/en/stable/images/AAPL.png) > **[See Apple's filings, financials, and insider trades on edgar.tools — no code required →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=quickstart) > ** You can access basic company data as properties: `company.industry # 'ELECTRONIC COMPUTERS' company.shares_outstanding # 15115785000.0 company.public_float # 2899948348000.0` Step 4: Get Financial Statements -------------------------------- This is the most common task — getting a company's financial statements: `financials = company.get_financials() # The three financial statements income = financials.income_statement() balance = financials.balance_sheet() cashflow = financials.cashflow_statement()` ![AAPL Income Statement](https://edgartools.readthedocs.io/en/stable/images/aapl-income-xbrl.webp) That's it — three lines to get any company's income statement, balance sheet, or cash flow. Common gotcha The canonical method is `cashflow_statement()`, but `cash_flow_statement()` also works. All three statements: `income_statement()`, `balance_sheet()`, `cashflow_statement()`. Step 5: Get Specific Values --------------------------- Need just one number instead of the full statement? `financials.get_revenue() # 391035000000 financials.get_net_income() # 93736000000` Step 6: Export to DataFrame --------------------------- Every financial statement converts to a pandas DataFrame for further analysis: `df = financials.income_statement().to_dataframe()` You can also export company filings: `filings = company.get_filings() df = filings.to_pandas()` Step 7: Browse Company Filings ------------------------------ Retrieve and filter a company's SEC filings: `# Get all filings filings = company.get_filings() # Filter by form type tenk_filings = company.get_filings(form="10-K") # Get the latest 10-K as a data object tenk = company.latest("10-K")` ![AAPL Filings](https://edgartools.readthedocs.io/en/stable/images/aapl-filings.png) How EdgarTools Is Organized --------------------------- Here's a map of the main objects. Use it as a reference when you want to try something new: `Company("AAPL") # Start here — look up a company ├── .get_financials() # Annual financials from 10-K (RECOMMENDED) │ ├── .income_statement() # Revenue, expenses, profit │ ├── .balance_sheet() # Assets, liabilities, equity │ ├── .cashflow_statement() # Cash in and out │ ├── .get_revenue() # Quick: just the revenue number │ └── .get_net_income() # Quick: just net income │ ├── .get_quarterly_financials() # Quarterly financials from 10-Q │ └── (same interface as above) │ ├── .get_filings(form="10-K") # Browse SEC filings │ ├── .head(5) # See the first 5 │ ├── .latest() # Get the most recent one │ └── [0].obj() # Parse into a data object (TenK, etc.) │ ├── .auditor # Auditor name, PCAOB ID, location │ └── .subsidiaries # Subsidiaries from Exhibit 21 │ └── .get_facts() # Historical data (for 4+ years of trends) ├── .income_statement() # Multi-year income data └── .balance_sheet() # Multi-year balance sheet` See it live on edgar.tools The code above runs locally. **[edgar.tools](https://app.edgar.tools/?utm_source=edgartools-docs&utm_medium=see-live&utm_content=general) ** puts the same data in a web UI with AI enrichment on top — no code required. * **[Browse Apple's filings, financials, and insider trades →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=general) ** * **[Watch filings arrive in real time →](https://app.edgar.tools/filings?utm_source=edgartools-docs&utm_medium=see-live&utm_content=general) ** * **[Search disclosures across 12 XBRL topics →](https://app.edgar.tools/disclosures?utm_source=edgartools-docs&utm_medium=see-live&utm_content=general) ** Also includes a REST API (20+ endpoints), hosted MCP server, and data exports. Free tier: 100 API calls/day. Step 8: Next Steps ------------------ You just learned how to install EdgarTools, look up a company, get financial statements, and browse filings. Here's where to go next: **Financial Data** * [Choosing the Right API](https://edgartools.readthedocs.io/en/stable/xbrl/getting-started/choosing-the-right-api/) — Which method to use for your task (start here!) * [Financial Statements Guide](https://edgartools.readthedocs.io/en/stable/guides/financial-data/) — Income statements, balance sheets, cash flow in depth * [Extract Statements from Filings](https://edgartools.readthedocs.io/en/stable/guides/extract-statements/) — XBRL data extraction * [Company Facts](https://edgartools.readthedocs.io/en/stable/guides/company-facts/) — Historical financial data across all filings **Companies & Filings** * [Find a Company](https://edgartools.readthedocs.io/en/stable/guides/finding-companies/) — Search by name, ticker, CIK, industry, or exchange * [Working with Filings](https://edgartools.readthedocs.io/en/stable/guides/working-with-filing/) — Open, view, and parse any SEC filing * [Search & Filter Filings](https://edgartools.readthedocs.io/en/stable/guides/searching-filings/) — Find exactly the filings you need **Filing Types** * [Annual & Quarterly Reports](https://edgartools.readthedocs.io/en/stable/concepts/data-objects/) — 10-K and 10-Q data objects * [Current Events (8-K)](https://edgartools.readthedocs.io/en/stable/eightk-filings/) — Material events and press releases * [Insider Trades (Form 4)](https://edgartools.readthedocs.io/en/stable/insider-filings/) — Monitor insider transactions * [Institutional Holdings (13F)](https://edgartools.readthedocs.io/en/stable/guides/thirteenf-data-object-guide/) — Who owns what **Reference** * [Cheat Sheet](https://edgartools.readthedocs.io/en/stable/quick-guide/) — Common operations at a glance * [Notebooks](https://edgartools.readthedocs.io/en/stable/notebooks/) — Interactive Colab tutorials you can run in your browser Getting Help ------------ * **[Documentation](https://edgartools.readthedocs.io/en/latest/) **: Browse our comprehensive guides * **[GitHub Discussions](https://github.com/dgunning/edgartools/discussions) **: Ask questions and share insights * **[Issues](https://github.com/dgunning/edgartools/issues) **: Report bugs or request features Support EdgarTools ------------------ If you found this quickstart helpful, consider supporting EdgarTools development: [![Buy Me A Coffee](https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png)](https://www.buymeacoffee.com/edgartools) Back to top --- # Track Insiders - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/company-insiders/#company-insiders-find-officers-directors-and-major-shareholders) Company Insiders: Find Officers, Directors, and Major Shareholders ================================================================== This guide shows how to get a list of insiders for a company by writing a simple script to loop through their Form 4 filings and getting the **name** and **position**. 1\. Deciding on an appropriate date range ----------------------------------------- The approach is to get all Form 4 Insider filings for the past 6 months. To specify the date range we use a use `timedelta` to subtract 6 months from `datetime.now()` `from datetime import datetime, timedelta from edgar import * date_range = ((datetime.now() - timedelta(days=6*30)) # Approximate 6 months .strftime('%Y-%m-%d:'))` 2\. Getting the company filings ------------------------------- Now we can use the `Company` class to get the company filings for the past 6 months. `c: Company = Company(ticker) filings: EntityFilings = c.get_filings(form='4', filing_date=date_range)` 3\. Collecting data from each Form 4 ------------------------------------ Now we loop through each filing and get the ownership summary, which contains the insider names and their positions. Each Form4 has an `OwnershipSummary` object that we can convert to a DataFrame. `dfs = [] # List to hold DataFrames for each filing for filing in tqdm(filings): form4: Form4 = filing.obj() summary = form4.get_ownership_summary() dfs.append(summary.to_dataframe()[['Insider', 'Position']])` 4\. Combining the DataFrames ---------------------------- Finally, we can concatenate all the DataFrames into a single DataFrame and drop duplicates to get a unique list of insiders. `import pandas as pd insiders = (pd.concat(dfs, ignore_index=True) .drop_duplicates().reset_index(drop=True) .sort_values(by='Position', key=lambda col: col == 'Director', ascending=True) )` 5\. Putting it all together --------------------------- The complete code to get the insiders for a company is as follows. Note that we put it inside a function so we can easily reuse it for different tickers. `import pandas as pd from rich import print from tqdm.auto import tqdm from edgar import * from edgar.entity import EntityFilings from edgar.ownership import Form4 from datetime import datetime, timedelta # Calculate the date 6 months ago from today date_range = ((datetime.now() - timedelta(days=6*30)) # Approximate 6 months .strftime('%Y-%m-%d:')) def get_insiders(ticker): c: Company = Company(ticker) filings: EntityFilings = c.get_filings(form='4', filing_date=date_range) dfs = [] for filing in tqdm(filings): form4: Form4 = filing.obj() summary = form4.get_ownership_summary() dfs.append(summary.to_dataframe()[['Insider', 'Position']]) insiders = (pd.concat(dfs, ignore_index=True) .drop_duplicates().reset_index(drop=True) .sort_values(by='Position', key=lambda col: col == 'Director', ascending=True) ) return insiders if __name__ == '__main__': insiders = get_insiders("NFLX") print(insiders)` See this on edgar.tools The script above loops through Form 4 filings to build an insider list for one company. **edgar.tools** has this pre-computed across 186K+ insider filings with 802K+ transactions — including net buy/sell sentiment and executive profiles. * **[See Netflix's insiders and transactions instantly →](https://app.edgar.tools/companies/NFLX?utm_source=edgartools-docs&utm_medium=see-live&utm_content=company-insiders) ** * **[See Apple's insider trading activity →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=company-insiders) ** No loops, no waiting. Free tier available. [Pricing →](https://app.edgar.tools/pricing?utm_source=edgartools-docs&utm_medium=see-live&utm_content=company-insiders) Back to top --- # Understanding SEC Filings - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/concepts/sec-filings/#understanding-sec-filings) Understanding SEC Filings ========================= Introduction ------------ The U.S. Securities and Exchange Commission (SEC) requires public companies, investment funds, and certain individuals to submit various regulatory filings. These documents provide transparency into financial performance, significant events, insider activities, and investment decisions. The SEC's Electronic Data Gathering, Analysis, and Retrieval system (EDGAR) makes these filings publicly available. This guide explains the key SEC filing types, their purposes, and how to access and analyze them using the `edgartools` library. Why SEC Filings Matter ---------------------- SEC filings are the most authoritative source of company information available to the public. Unlike press releases, investor presentations, or news articles, SEC filings: * Are legally required to be accurate and complete * Follow standardized formats for consistency * Contain detailed financial data and disclosures * Are subject to strict liability for false or misleading information * Provide a historical record of a company's development Common SEC Filing Types ----------------------- ### Company Reporting Forms | Form | Description | Frequency | Key Information | | --- | --- | --- | --- | | **10-K** | Annual report | Annual | Comprehensive financial statements, business description, risk factors, management discussion | | **10-Q** | Quarterly report | Quarterly | Interim financial statements, updates since last 10-K | | **8-K** | Current report | As needed | Material events (acquisitions, executive changes, bankruptcy) | | **S-1** | Registration statement | Before IPO | Business model, financials, risk factors, use of proceeds | | **DEF 14A** | Proxy statement | Annual | Executive compensation, board members, shareholder proposals | ### Ownership and Investment Forms | Form | Description | Filed By | Key Information | | --- | --- | --- | --- | | **Form 3** | Initial ownership | Insiders | Initial positions when becoming an insider | | **Form 4** | Changes in ownership | Insiders | Purchases, sales, and other transactions | | **Form 5** | Annual ownership | Insiders | Summary of transactions for the year | | **13F** | Holdings report | Investment funds | Portfolio holdings of investment managers | | **13D/G** | Beneficial ownership | 5%+ shareholders | Significant ownership positions and intentions | Anatomy of Key Filings ---------------------- ### 10-K Annual Report The 10-K is the most comprehensive filing and typically contains: 1. **Business Overview** (Part I, Item 1) 2. Company operations, products/services, markets 3. Revenue breakdown by segment 4. Competitive landscape 5. **Risk Factors** (Part I, Item 1A) 6. Detailed disclosure of business risks 7. Industry, operational, and financial risks 8. **Management's Discussion & Analysis** (Part II, Item 7) 9. Analysis of financial condition and results 10. Liquidity and capital resources 11. Critical accounting policies 12. **Financial Statements** (Part II, Item 8) 13. Balance sheet 14. Income statement 15. Cash flow statement 16. Statement of shareholders' equity 17. Notes to financial statements 18. **Controls and Procedures** (Part II, Item 9) 19. Disclosure controls 20. Internal control over financial reporting ### 10-Q Quarterly Report The 10-Q is a condensed version of the 10-K filed quarterly, containing: * Unaudited financial statements * Management's discussion of results * Updates on risk factors * Disclosure of material events ### 8-K Current Report The 8-K reports significant events that occur between 10-K and 10-Q filings: * Item 1.01: Entry into a Material Agreement * Item 2.01: Completion of Acquisition or Disposition * Item 5.02: Departure/Election of Directors or Officers * Item 7.01: Regulation FD Disclosure * Item 8.01: Other Events ### Form 4 (Insider Transactions) Form 4 discloses transactions by company insiders (directors, officers, 10%+ shareholders): * Transaction date and type (purchase, sale, grant, exercise) * Number of securities involved * Price per share * Resulting ownership after transaction ### 13F (Investment Fund Holdings) 13F reports show investment portfolios of funds managing over $100 million: * Securities held at quarter-end * Number of shares * Market value * Investment discretion Working with SEC Filings in edgartools -------------------------------------- ### Accessing Filings `from edgar import Company # Get all filings for a specific company apple = Company("AAPL") filings = apple.get_filings() # Filter by form type annual_reports = apple.get_filings(form="10-K") quarterly_reports = apple.get_filings(form="10-Q") current_reports = apple.get_filings(form="8-K") # Get the most recent annual report latest_10k = annual_reports.latest()` ### Extracting Financial Data The simplest way to get financial statements is through the `get_financials()` method on a Company: `# Get financial statements (recommended approach) company = Company("AAPL") financials = company.get_financials() # Access specific statements balance_sheet = financials.balance_sheet() income_stmt = financials.income_statement() cash_flow = financials.cashflow_statement() # Get specific values directly revenue = financials.get_revenue() net_income = financials.get_net_income() total_assets = financials.get_total_assets()` ### Analyzing Insider Trading `from edgar import Company # Get Form 4 filings for a company tesla = Company("TSLA") form4_filings = tesla.get_filings(form=4) # Parse the most recent filing form4 = form4_filings.latest(1).obj() # Get the transaction summary summary = form4.get_ownership_summary() print(f"Insider: {summary.insider_name}") print(f"Position: {summary.position}") print(f"Activity: {summary.primary_activity}") print(f"Net shares changed: {summary.net_change}")` ### Researching Investment Fund Holdings `from edgar import get_filings # Get 13F filings (institutional holdings reports) thirteenf_filings = get_filings(form="13F-HR") # Parse a specific fund's holdings filing = thirteenf_filings[0] thirteenf = filing.obj() # View top holdings print(thirteenf.holdings)` Best Practices for Working with SEC Filings ------------------------------------------- ### 1\. Understand Filing Timelines * **10-K**: Due 60-90 days after fiscal year-end (depending on company size) * **10-Q**: Due 40-45 days after quarter-end * **8-K**: Due within 4 business days of the event * **Form 4**: Due within 2 business days of the transaction * **13F**: Due within 45 days of quarter-end ### 2\. Be Aware of Filing Amendments Amendments are indicated with a suffix: - 10-K/A, 10-Q/A, 8-K/A, etc. `# Get original and amended filings filings = company.get_filings(form="10-K") amendments = filings.filter(form="10-K/A")` ### 3\. Handle Historical Data Carefully * Financial restatements can change historical data * Company structures change over time (mergers, spin-offs) * Accounting standards evolve ### 4\. Respect SEC Access Guidelines The SEC has rate limits for EDGAR access: - Identify yourself properly with `edgar.set_identity()` - Implement appropriate delays between requests - Consider using local caching for repeated access `from edgar import set_identity # Set your identity for SEC access set_identity("Your Name your.email@example.com")` Conclusion ---------- SEC filings provide a wealth of structured and unstructured data for financial analysis, investment research, and regulatory compliance. With `edgartools`, you can efficiently access, parse, and analyze these filings to extract valuable insights. Understanding the different filing types, their purposes, and how to work with them programmatically allows you to build sophisticated financial analysis workflows and make more informed investment decisions. Additional Resources -------------------- * [SEC EDGAR Website](https://www.sec.gov/edgar/search-and-access) * [SEC Filing Deadlines](https://www.sec.gov/edgar/filer-information/calendar) * [EDGAR Filing Codes](https://www.sec.gov/info/edgar/forms/edgform.pdf) Back to top --- # Common Pitfalls - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/common-pitfalls/#common-pitfalls) Common Pitfalls =============== New to EdgarTools? This page covers the most frequent mistakes and how to avoid them. * * * Which API Should I Use? ----------------------- EdgarTools has three ways to get financial data. This causes the most confusion for new users. | I want to... | Use this | Not this | | --- | --- | --- | | Get revenue, net income, balance sheet | `company.get_financials()` | `company.get_facts()` or `filing.xbrl()` | | Compare Apple vs Microsoft financials | `company.get_financials()` for each | Manual XBRL parsing | | Get 5+ years of historical trends | `company.get_facts()` | Iterating over 5 filings | | Get segment breakdowns or footnotes | `filing.xbrl()` | `get_financials()` (no segments) | **Rule of thumb:** Start with `get_financials()`. It covers 95% of use cases. Only use `get_facts()` for 4+ years of history, and `filing.xbrl()` for segments/footnotes. For a detailed comparison, see [Choosing the Right API](https://edgartools.readthedocs.io/en/latest/xbrl/getting-started/choosing-the-right-api/) . * * * Financial Data -------------- ### Use `get_financials()`, not `filing.xbrl()` `# WRONG: Too complex for simple financial data filing = Company("AAPL").get_filings(form="10-K").latest() xbrl = filing.xbrl() statements = xbrl.statements income = statements.income_statement() # RIGHT: Direct and simple financials = Company("AAPL").get_financials() income = financials.income_statement()` ### Use `get_financials()`, not `get_facts()` for standard data `# WRONG: get_facts() is for historical trends, not everyday use facts = company.get_facts() income = facts.income_statement(periods=3, annual=True) # RIGHT: get_financials() already includes 3 years financials = company.get_financials() income = financials.income_statement() # Already 3 years revenue = financials.get_revenue() # Single current value` ### Quick getters can return `None` Always check before doing math: `revenue = financials.get_revenue() net_income = financials.get_net_income() # WRONG: Will crash if either is None margin = net_income / revenue # RIGHT: Check first if revenue and net_income: margin = net_income / revenue` ### `period_offset=0` means current year, not prior year `this_year = financials.get_revenue() # period_offset=0 (default) last_year = financials.get_revenue(period_offset=1) # 1 year ago two_ago = financials.get_revenue(period_offset=2) # 2 years ago` ### Financial values are in actual dollars Values from `get_financials()` are in real dollars, not thousands or millions: `revenue = financials.get_revenue() # e.g. 391035000000 print(f"${revenue/1e9:.1f}B") # "$391.0B"` ### The cash flow method is `cashflow_statement()` The canonical method name is `cashflow_statement()` (no underscore between "cash" and "flow"). The alias `cash_flow_statement()` also works. All three statements follow the same pattern: `financials.income_statement() # Income statement financials.balance_sheet() # Balance sheet financials.cashflow_statement() # Cash flow statement (cash_flow_statement() also works)` * * * Filings ------- ### Always limit before iterating `# WRONG: Loads ALL filings into memory for filing in company.get_filings(): process(filing) # RIGHT: Limit first for filing in company.get_filings(form="10-K").head(10): process(filing.obj())` ### Use `.head(n)`, not `list(filings)[:n]` `# WRONG: Converts ALL filings to a list first recent = list(company.get_filings())[:5] # RIGHT: Only fetches 5 recent = company.get_filings().head(5)` ### Use `filing.obj()` for sections, not `filing.document()` `# WRONG: Raw document text, no structure doc = filing.document() text = doc.text() # RIGHT: Parsed data object with sections tenk = filing.obj() risk_factors = tenk['Item 1A'] business = tenk['Item 1']` * * * Insider Trading (Form 4) ------------------------ ### Use `get_ownership_summary()`, not raw DataFrames `summary = form4.get_ownership_summary() print(f"Activity: {summary.primary_activity}") # "Purchase", "Sale", "Mixed" print(f"Net shares: {summary.net_change:,}") # positive=buy, negative=sell print(f"Net value: ${summary.net_value:,.0f}")` ### `get_ownership_summary()` returns ONE object, not a list `# WRONG: Will crash — summary is not iterable summary = form4.get_ownership_summary() for item in summary: # TypeError! print(item.insider_name) # RIGHT: Access properties directly summary = form4.get_ownership_summary() summary.insider_name # "Tim Cook" summary.primary_activity # "Sale" # To process MULTIPLE filings, loop over filings: for filing in company.get_filings(form="4").head(20): summary = filing.obj().get_ownership_summary() print(f"{summary.insider_name}: {summary.primary_activity}")` ### Transaction codes have specific meanings | Code | Meaning | Signal | | --- | --- | --- | | P | Open market purchase | Strong buy signal | | S | Open market sale | Sell signal | | A | Grant/Award | Compensation, not a purchase | | M | Option exercise | Converting options, not buying | | F | Tax withholding | Shares sold to cover taxes | The `primary_activity` property on `TransactionSummary` interprets these for you. * * * Institutional Holdings (13F) ---------------------------- ### Holdings value is in $1,000s, not dollars `holdings = thirteenf.holdings # WRONG: Off by 1000x total = holdings['Value'].sum() # RIGHT: Multiply by 1000 total = holdings['Value'].sum() * 1000` ### Use CUSIP for reliable lookups, not ticker `# Ticker may be missing for some securities apple = holdings[holdings['Cusip'] == '037833100'] # Reliable apple = holdings[holdings['Ticker'] == 'AAPL'] # May miss some` ### Don't iterate all 13Fs to find holders of a stock `# WRONG: Iterating 8,000+ filings to find "who holds Apple?" for filing in get_filings(form="13F-HR"): holdings = filing.obj().holdings if 'AAPL' in holdings['Ticker'].values: print(filing.company) # RIGHT: Filter by filer first filing = get_filings(form="13F-HR").filter(company="BERKSHIRE")[0] holdings = filing.obj().holdings` ### 13F filings have a 45-day lag Holdings are reported as of quarter-end, but filings aren't due for 45 days. Q3 (Sep 30) data may not appear until mid-November. * * * 10-K / 10-Q / 8-K Reports ------------------------- ### Item keys differ between report types `# 10-K / 10-Q: "Item 1", "Item 1A", "Item 7" tenk['Item 1A'] # Risk Factors # 8-K: "Item 1.01", "Item 5.02" (with dots) eightk['Item 5.02'] # Personnel Changes` ### Not all items exist in every filing `risk_factors = tenk.risk_factors if risk_factors: print(risk_factors.text[:1000]) else: print("No risk factors section in this filing")` * * * Exporting Data -------------- ### Export statements, not the financials object `financials = company.get_financials() # WRONG: financials object has no to_dataframe() df = financials.to_dataframe() # AttributeError # RIGHT: call to_dataframe() on the individual statement df = financials.income_statement().to_dataframe() df = financials.balance_sheet().to_dataframe() df = financials.cashflow_statement().to_dataframe()` ### Export to CSV or Excel `income = company.get_financials().income_statement() # CSV income.to_dataframe().to_csv("income.csv") # Excel income.to_dataframe().to_excel("income.xlsx")` ### Export filings list (not financial data) `# Export a list of filings to a DataFrame filings = company.get_filings(form="10-K") df = filings.to_pandas() # One row per filing df.to_csv("10k_filings.csv")` ### Financial values are raw numbers (not formatted) `get_revenue()` returns an integer like `391035000000`, not `"$391B"`. Format it yourself: `revenue = financials.get_revenue() print(f"${revenue / 1e9:.1f}B") # "$391.0B"` * * * XBRL ---- ### `filing.xbrl()` can return `None` Not all filings have XBRL data. Always check: `xbrl = filing.xbrl() if xbrl: print(xbrl.entity_name) else: print("No XBRL data in this filing")` ### XBRL concept names vary by company Different companies use different concept names for the same thing (e.g., "Revenues" vs "RevenueFromContractWithCustomerExcludingAssessedTax"). Use `get_financials()` instead, which normalizes these automatically. `# Instead of guessing concept names: revenue = company.get_financials().get_revenue()` Back to top --- # Configuration - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/configuration/#configuration) Configuration ============= EdgarTools provides extensive configuration options through environment variables and programmatic settings to customize behavior, optimize performance, and ensure SEC compliance. Quick Setup ----------- For most users, you only need to set your identity: `export EDGAR_IDENTITY="Your Name your.email@company.com"` Or in Python: `from edgar import set_identity set_identity("Your Name your.email@company.com")` Environment Variables --------------------- ### Required Configuration #### EDGAR\_IDENTITY **Required for all SEC requests** Sets the User-Agent string for SEC EDGAR requests. Required by SEC to identify your application. `export EDGAR_IDENTITY="John Doe john.doe@company.com"` **Format Options:** - `"Name email@domain.com"` - Full name and email (recommended) - `"email@domain.com"` - Email only (acceptable) - `"Company Name contact@company.com"` - Company identification **Python Alternative:** `from edgar import set_identity set_identity("John Doe john.doe@company.com")` ### Performance and Access Control #### EDGAR\_ACCESS\_MODE Controls HTTP request behavior and connection limits to manage SEC server load. `export EDGAR_ACCESS_MODE="NORMAL"` **Available Modes:** | Mode | Timeout | Max Connections | Retries | Use Case | | --- | --- | --- | --- | --- | | `NORMAL` | 15s | 10 | 3 | Default - balanced performance | | `CAUTION` | 20s | 5 | 3 | Conservative - reduces server load | | `CRAWL` | 25s | 2 | 2 | Minimal impact - bulk processing | **Examples:** `# High-performance research (default) export EDGAR_ACCESS_MODE="NORMAL" # Conservative access for production export EDGAR_ACCESS_MODE="CAUTION" # Bulk data processing with minimal server impact export EDGAR_ACCESS_MODE="CRAWL"` ### Local Data Storage #### EDGAR\_USE\_LOCAL\_DATA Enables local caching of SEC data for improved performance and reduced API calls. `export EDGAR_USE_LOCAL_DATA="True"` **Values:** - `"True"`, `"true"`, `"1"` - Enable local storage - `"False"`, `"false"`, `"0"` - Disable local storage (default) **Benefits of Local Storage:** - Faster repeated access to same data - Reduced SEC API calls - Offline access to cached data - Better performance for bulk operations **Python Alternative:** `from edgar import use_local_storage use_local_storage(True)` #### EDGAR\_LOCAL\_DATA\_DIR Sets the directory for local data storage. `export EDGAR_LOCAL_DATA_DIR="/path/to/your/edgar/data"` **Default:** `~/.edgar` (user's home directory) **Directory Structure:** `~/.edgar/ # Root data directory ├── requestcache/ # HTTP response cache ├── filings/ # Downloaded filing data │ └── YYYYMMDD/ # Organized by date ├── submissions/ # Company submissions data ├── companyfacts/ # Company facts data └── reference/ # Reference data (tickers, etc.)` **Example Setup:** `# Custom data directory for project export EDGAR_LOCAL_DATA_DIR="/project/edgar_data" export EDGAR_USE_LOCAL_DATA="True"` ### Security and SSL #### EDGAR\_VERIFY\_SSL Controls SSL certificate verification for HTTPS requests. `export EDGAR_VERIFY_SSL="true"` **Values:** - `"true"` (default) - Verify SSL certificates (recommended) - `"false"`, `"0"`, `"no"`, `"n"`, `"off"` - Disable SSL verification **⚠️ Security Warning:** Only disable SSL verification in controlled environments. This reduces security by allowing man-in-the-middle attacks. **Use Cases for Disabling:** - Corporate proxy environments with custom certificates - Development environments with self-signed certificates - Network environments with SSL inspection ### HTTP Rate Limiting Rate limiting is implemented in `httpclient_ratelimiting`. The default rate limit is 9 requests per second. SEC has a maximum of 10 requests per second. To change the rate limit, call: `httpclient.update_rate_limiter(requests_per_second: int)`. ### Advanced: Distributed Rate Limiting Distributed Rate Limiting: rate limiting is implemented using [pyrate\_limiter](https://pypi.org/project/pyrate-limiter/) . To use a distributed rate limiter, such as for multiprocessing, define an httpclient.\_RATE\_LIMITER. See the pyrate\_limiter documentation and examples for details. Enterprise Configuration ------------------------ EdgarTools v4.28.0+ supports enterprise-grade configuration for custom SEC data sources and flexible rate limiting, enabling deployment with private mirrors, academic institutions, and high-volume applications. ### Configurable Rate Limiting #### EDGAR\_RATE\_LIMIT\_PER\_SEC Control the maximum number of requests per second to SEC servers or custom mirrors. `export EDGAR_RATE_LIMIT_PER_SEC="9"` **Default:** `9` requests/second (SEC's official limit) **Typical Values:** - `9` - SEC's standard limit (default, recommended for official SEC servers) - `10` - SEC's maximum allowed limit - Higher values (e.g., `20`, `50`) - Only for authorized custom mirrors with relaxed rate restrictions **Use Cases:** - **Custom mirrors**: Higher rate limits for private infrastructure with different restrictions - **Authorized applications**: High-volume applications with special SEC authorization - **Testing environments**: Flexible rate limits for development and testing - **International mirrors**: Optimized rates for regional mirrors **Example:** `# Standard SEC access (default) export EDGAR_RATE_LIMIT_PER_SEC="9" # Custom mirror with relaxed limits export EDGAR_RATE_LIMIT_PER_SEC="50" export EDGAR_BASE_URL="https://sec-mirror.company.com"` **Python Alternative:** `from edgar import httpclient # Update rate limiter programmatically httpclient.update_rate_limiter(requests_per_second=20)` **⚠️ Important:** Only use rate limits higher than 10 req/sec with custom mirrors or when authorized by SEC. Exceeding SEC's 10 req/sec limit may result in IP blocking. ### Custom SEC Data Sources Configure EdgarTools to use custom SEC mirrors, private data sources, or testing servers instead of the official SEC website. #### EDGAR\_BASE\_URL Sets the base URL for SEC EDGAR website access. `export EDGAR_BASE_URL="https://www.sec.gov"` **Default:** `https://www.sec.gov` **Use Cases:** - Corporate SEC mirrors for compliance workflows - Academic research institutions with local mirrors - Regional mirrors for reduced latency (international users) - Testing environments with mock servers **Example:** `# Corporate mirror export EDGAR_BASE_URL="https://sec-mirror.company.com" # Academic institution mirror export EDGAR_BASE_URL="https://sec.university.edu" # Regional mirror (example) export EDGAR_BASE_URL="https://sec-eu.example.com"` #### EDGAR\_DATA\_URL Sets the base URL for SEC data archives (filing documents, submissions, company facts). `export EDGAR_DATA_URL="https://data.sec.gov"` **Default:** `https://data.sec.gov` **Use Cases:** - Separate data server from website server - CDN acceleration for filing downloads - Private data repositories - Bandwidth optimization **Example:** `# Use CDN for data downloads export EDGAR_DATA_URL="https://cdn.sec-data.company.com" # Corporate data repository export EDGAR_DATA_URL="https://sec-data.company.com"` #### EDGAR\_XBRL\_URL Sets the base URL for XBRL-specific data and services. `export EDGAR_XBRL_URL="https://www.sec.gov"` **Default:** `https://www.sec.gov` **Use Cases:** - Specialized XBRL processing servers - XBRL validation and parsing services - Enhanced XBRL data repositories **Example:** `# Dedicated XBRL server export EDGAR_XBRL_URL="https://xbrl.sec-mirror.company.com"` ### Complete Enterprise Configuration Example #### Corporate Mirror Setup `# Corporate SEC mirror with higher rate limits export EDGAR_IDENTITY="Corporate Compliance compliance@company.com" export EDGAR_BASE_URL="https://sec-mirror.company.com" export EDGAR_DATA_URL="https://sec-data.company.com" export EDGAR_XBRL_URL="https://sec-xbrl.company.com" export EDGAR_RATE_LIMIT_PER_SEC="50" export EDGAR_ACCESS_MODE="NORMAL" export EDGAR_USE_LOCAL_DATA="True" export EDGAR_LOCAL_DATA_DIR="/var/lib/edgar"` #### Academic Research Institution `# University research mirror with custom rate limits export EDGAR_IDENTITY="Research Lab research@university.edu" export EDGAR_BASE_URL="https://sec.university.edu" export EDGAR_DATA_URL="https://sec-data.university.edu" export EDGAR_RATE_LIMIT_PER_SEC="25" export EDGAR_USE_LOCAL_DATA="True" export EDGAR_LOCAL_DATA_DIR="/research/edgar_data"` #### Regional Mirror (International Users) `# Regional mirror for reduced latency export EDGAR_IDENTITY="International Analyst analyst@company.com" export EDGAR_BASE_URL="https://sec-eu.example.com" export EDGAR_DATA_URL="https://sec-data-eu.example.com" export EDGAR_RATE_LIMIT_PER_SEC="15" export EDGAR_ACCESS_MODE="NORMAL"` #### Development/Testing Environment `# Mock SEC server for testing export EDGAR_IDENTITY="Developer dev@company.com" export EDGAR_BASE_URL="http://localhost:8080" export EDGAR_DATA_URL="http://localhost:8080/data" export EDGAR_XBRL_URL="http://localhost:8080/xbrl" export EDGAR_RATE_LIMIT_PER_SEC="100" # No limits for testing export EDGAR_VERIFY_SSL="false" # Self-signed certificates in dev export EDGAR_USE_RICH_LOGGING="1"` ### Python Configuration API Configure enterprise settings programmatically: `import os # Set custom SEC mirror os.environ['EDGAR_BASE_URL'] = "https://sec-mirror.company.com" os.environ['EDGAR_DATA_URL'] = "https://sec-data.company.com" os.environ['EDGAR_RATE_LIMIT_PER_SEC'] = "50" # Now import and use EdgarTools from edgar import Company company = Company("AAPL") # Uses custom mirror filings = company.get_filings(form="10-K")` **Note:** Environment variables must be set before importing EdgarTools modules, as configuration is evaluated at import time. ### Docker/Container Configuration For containerized deployments with custom SEC mirrors: `# Dockerfile FROM python:3.11-slim # Install EdgarTools RUN pip install edgartools # Configure enterprise SEC access ENV EDGAR_IDENTITY="Container App app@company.com" ENV EDGAR_BASE_URL="https://sec-mirror.company.com" ENV EDGAR_DATA_URL="https://sec-data.company.com" ENV EDGAR_RATE_LIMIT_PER_SEC="50" ENV EDGAR_ACCESS_MODE="CAUTION" ENV EDGAR_USE_LOCAL_DATA="True" ENV EDGAR_LOCAL_DATA_DIR="/app/edgar_data" # Create data directory RUN mkdir -p /app/edgar_data VOLUME /app/edgar_data WORKDIR /app` **Docker Compose Example:** `version: '3.8' services: edgar-app: image: your-edgar-app:latest environment: - EDGAR_IDENTITY=Service app@company.com - EDGAR_BASE_URL=https://sec-mirror.company.com - EDGAR_DATA_URL=https://sec-data.company.com - EDGAR_RATE_LIMIT_PER_SEC=50 - EDGAR_USE_LOCAL_DATA=True - EDGAR_LOCAL_DATA_DIR=/data volumes: - edgar-data:/data volumes: edgar-data:` ### Validation and Testing Verify your enterprise configuration: `import os from edgar import Company def validate_enterprise_config(): """Validate enterprise EdgarTools configuration.""" print("Enterprise Configuration:") print(f" Base URL: {os.getenv('EDGAR_BASE_URL', 'https://www.sec.gov')}") print(f" Data URL: {os.getenv('EDGAR_DATA_URL', 'https://data.sec.gov')}") print(f" XBRL URL: {os.getenv('EDGAR_XBRL_URL', 'https://www.sec.gov')}") print(f" Rate Limit: {os.getenv('EDGAR_RATE_LIMIT_PER_SEC', '9')} req/sec") # Test basic functionality try: company = Company("AAPL") print(f"\n✓ Successfully connected: {company.name}") # Test filing access filings = company.get_filings(form="10-K").head(1) if filings: print(f"✓ Successfully retrieved filings from: {filings[0].accession_number}") return True except Exception as e: print(f"\n❌ Configuration test failed: {e}") return False # Run validation validate_enterprise_config()` ### Troubleshooting Enterprise Configuration #### Custom Mirror Connection Issues `# Test connectivity to custom mirror import requests base_url = os.getenv('EDGAR_BASE_URL') try: response = requests.get(f"{base_url}/cgi-bin/browse-edgar") print(f"✓ Mirror accessible: {response.status_code}") except Exception as e: print(f"❌ Mirror connection failed: {e}")` #### Rate Limit Verification `# Verify rate limiter is using correct setting from edgar import httpclient print(f"Current rate limit: {os.getenv('EDGAR_RATE_LIMIT_PER_SEC', '9')} req/sec") # Monitor rate limiting in action import time start = time.time() for i in range(20): # Make 20 requests company = Company("AAPL") elapsed = time.time() - start actual_rate = 20 / elapsed print(f"Actual request rate: {actual_rate:.2f} req/sec")` #### SSL Certificate Issues with Custom Mirrors `# If custom mirror uses self-signed certificates export EDGAR_VERIFY_SSL="false" # Or configure SSL certificate bundle export REQUESTS_CA_BUNDLE="/path/to/company-ca-bundle.crt"` ### Best Practices for Enterprise Deployment 1. **Always set EDGAR\_IDENTITY** - Include company/team identification 2. **Test mirror connectivity** - Validate URLs before production deployment 3. **Monitor rate limits** - Ensure compliance with mirror's rate restrictions 4. **Use local data storage** - Enable caching for improved performance 5. **Secure credentials** - Use environment variables, not hardcoded values 6. **Document configuration** - Maintain configuration profiles for different environments 7. **Version control** - Use `.env.example` files to document required variables 8. **Health checks** - Implement validation functions to verify configuration ### Environment Variables Summary | Variable | Default | Purpose | Enterprise Use Case | | --- | --- | --- | --- | | `EDGAR_RATE_LIMIT_PER_SEC` | `9` | Request rate limit | Custom mirrors, authorized high-volume apps | | `EDGAR_BASE_URL` | `https://www.sec.gov` | SEC website base URL | Corporate mirrors, regional mirrors | | `EDGAR_DATA_URL` | `https://data.sec.gov` | Data archives URL | CDN acceleration, private repositories | | `EDGAR_XBRL_URL` | `https://www.sec.gov` | XBRL services URL | Specialized XBRL servers | ### Backward Compatibility All enterprise configuration features are **fully backward compatible**: - Default values point to official SEC servers - Zero configuration needed for standard users - Existing code continues to work without changes - Environment variables are optional ### HTTP Caching Web requests are cached by default, according to the rules defined in httpclient\_cache. #### Cache Directory The cache directory is set in `httpclient.CACHE_DIRECTORY`, set to `_cache` by default. Set CACHE\_DIRECTORY=None to disable cache. Call `httpclient.close_client()` after any changes to the CACHE\_DIRECTORY variable. #### Caching Rules The SEC marks all requests as either NO-STORE or NO-CACHE, therefore a custom cache controller was implemented with the following rules: - `/submissions` URLs for up to 10 minutes by default, set in `MAX_SUBMISSIONS_AGE_SECONDS` - `.*index/.*` URLs for up to 30 minutes by default, set in `MAX_INDEX_AGE_SECONDS` - `/Archives/edgar/data` URLs indefinitely (forever) See `httpclient_cache` for implementation. #### Advanced: Alternative Storage Caches * The underlying cache uses FileCache for local file storage. Alternative storage backends may be available through httpxthrottlecache configuration. See https://github.com/paultiq/httpxthrottlecache for details. #### EDGAR\_USE\_RICH\_LOGGING Enables enhanced console logging with rich formatting. `export EDGAR_USE_RICH_LOGGING="1"` **Values:** - `"1"` - Enable rich logging with colors and formatting - `"0"` (default) - Standard logging **Benefits:** - Color-coded log levels - Enhanced readability - Progress bars and status indicators - Better debugging information Programmatic Configuration -------------------------- ### Setting Identity `from edgar import set_identity # Set identity programmatically set_identity("Research Team research@university.edu") # Verify identity is set from edgar.core import get_identity print(f"Current identity: {get_identity()}")` ### Local Storage Control `from edgar import use_local_storage # Enable local storage use_local_storage(True) # Disable local storage use_local_storage(False) # Check current setting from edgar.storage import using_local_storage print(f"Using local storage: {using_local_storage()}")` ### HTTP Client Configuration `from edgar.core import EdgarSettings # Custom access mode custom_settings = EdgarSettings( http_timeout=30, # 30 second timeout max_connections=3, # Maximum 3 concurrent connections retries=5 # 5 retry attempts ) # Apply custom settings (requires restarting client)` Configuration Profiles ---------------------- ### Research Profile Optimized for interactive research and analysis: `export EDGAR_IDENTITY="Researcher researcher@university.edu" export EDGAR_ACCESS_MODE="NORMAL" export EDGAR_USE_LOCAL_DATA="True" export EDGAR_USE_RICH_LOGGING="1"` ### Production Profile Conservative settings for production environments: `export EDGAR_IDENTITY="Production System api@company.com" export EDGAR_ACCESS_MODE="CAUTION" export EDGAR_USE_LOCAL_DATA="True" export EDGAR_LOCAL_DATA_DIR="/var/lib/edgar" export EDGAR_VERIFY_SSL="true"` ### Bulk Processing Profile Minimal server impact for large-scale data processing: `export EDGAR_IDENTITY="Bulk Processor batch@company.com" export EDGAR_ACCESS_MODE="CRAWL" export EDGAR_USE_LOCAL_DATA="True" export EDGAR_LOCAL_DATA_DIR="/data/edgar"` ### Development Profile Flexible settings for development and testing: `export EDGAR_IDENTITY="Developer dev@company.com" export EDGAR_ACCESS_MODE="NORMAL" export EDGAR_USE_LOCAL_DATA="True" export EDGAR_USE_RICH_LOGGING="1" export EDGAR_VERIFY_SSL="false" # Only if needed for proxy` ### Enterprise Mirror Profile For custom SEC mirrors with higher rate limits (see [Enterprise Configuration](https://edgartools.readthedocs.io/en/stable/configuration/#enterprise-configuration) ): `export EDGAR_IDENTITY="Corporate Compliance compliance@company.com" export EDGAR_BASE_URL="https://sec-mirror.company.com" export EDGAR_DATA_URL="https://sec-data.company.com" export EDGAR_RATE_LIMIT_PER_SEC="50" export EDGAR_USE_LOCAL_DATA="True" export EDGAR_LOCAL_DATA_DIR="/var/lib/edgar"` Configuration File Setup ------------------------ ### .env File Create a `.env` file in your project root: `# .env file EDGAR_IDENTITY=John Doe john.doe@company.com EDGAR_ACCESS_MODE=NORMAL EDGAR_USE_LOCAL_DATA=True EDGAR_LOCAL_DATA_DIR=./edgar_data EDGAR_USE_RICH_LOGGING=1` Load with python-dotenv: `from dotenv import load_dotenv load_dotenv() # Now EdgarTools will use the environment variables from edgar import Company company = Company("AAPL")` ### Shell Configuration Add to your shell profile (`.bashrc`, `.zshrc`, etc.): `# Edgar Tools Configuration export EDGAR_IDENTITY="Your Name your.email@company.com" export EDGAR_ACCESS_MODE="NORMAL" export EDGAR_USE_LOCAL_DATA="True" export EDGAR_LOCAL_DATA_DIR="$HOME/.edgar"` Data Management --------------- ### Local Storage Benefits When `EDGAR_USE_LOCAL_DATA="True"`: 1. **Caching**: HTTP responses cached locally 2. **Offline Access**: Previously accessed data available offline 3. **Performance**: Faster subsequent access to same data 4. **Reduced API Calls**: Less load on SEC servers ### Storage Space Considerations Typical storage usage: - **Company submissions**: ~100MB for major companies - **Company facts**: ~50MB for major companies \- **HTTP cache**: Varies based on usage - **Individual filings**: 1-10MB each Troubleshooting Configuration ----------------------------- ### Check Current Configuration `import os from edgar.core import get_identity # Check identity print(f"Identity: {get_identity()}") # Check access mode print(f"Access Mode: {os.getenv('EDGAR_ACCESS_MODE', 'NORMAL')}") # Check local data settings print(f"Use Local Data: {os.getenv('EDGAR_USE_LOCAL_DATA', 'False')}") print(f"Data Directory: {os.getenv('EDGAR_LOCAL_DATA_DIR', '~/.edgar')}") # Check SSL verification print(f"Verify SSL: {os.getenv('EDGAR_VERIFY_SSL', 'true')}")` ### Common Issues #### Identity Not Set `# Error: No identity set # Solution: set_identity("Your Name your.email@company.com")` #### Permission Errors `# Error: Permission denied writing to ~/.edgar # Solution: Check directory permissions or use custom directory export EDGAR_LOCAL_DATA_DIR="/tmp/edgar"` #### SSL Verification Errors `# Error: SSL certificate verification failed # Solution: Disable SSL verification (only if safe) export EDGAR_VERIFY_SSL="false"` #### Connection Timeouts `# Error: Connection timeouts in slow network # Solution: Use more conservative settings export EDGAR_ACCESS_MODE="CAUTION"` Security Best Practices ----------------------- 1. **Always set EDGAR\_IDENTITY** - Required for SEC compliance 2. **Keep SSL verification enabled** - Only disable in controlled environments 3. **Secure data directory** - Ensure appropriate file permissions 4. **Use least-privilege access** - Don't run with unnecessary elevated permissions 5. **Monitor data usage** - Be aware of local storage space consumption Docker Configuration -------------------- For containerized deployments: `# Dockerfile ENV EDGAR_IDENTITY="Container App app@company.com" ENV EDGAR_ACCESS_MODE="CAUTION" ENV EDGAR_USE_LOCAL_DATA="True" ENV EDGAR_LOCAL_DATA_DIR="/app/edgar_data" # Create data directory RUN mkdir -p /app/edgar_data VOLUME /app/edgar_data` Configuration Validation ------------------------ Validate your configuration before running analysis: `from edgar import Company import os def validate_config(): """Validate EdgarTools configuration.""" issues = [] # Check identity try: from edgar.core import get_identity identity = get_identity() if not identity: issues.append("EDGAR_IDENTITY not set") elif "@" not in identity: issues.append("EDGAR_IDENTITY should include email") except: issues.append("Cannot retrieve EDGAR_IDENTITY") # Check data directory if os.getenv('EDGAR_USE_LOCAL_DATA', 'False').lower() in ['true', '1']: data_dir = os.getenv('EDGAR_LOCAL_DATA_DIR', '~/.edgar') expanded_dir = os.path.expanduser(data_dir) if not os.path.exists(expanded_dir): try: os.makedirs(expanded_dir, exist_ok=True) except: issues.append(f"Cannot create data directory: {data_dir}") # Test basic functionality try: company = Company("AAPL") print(f"✓ Successfully created company: {company.name}") except Exception as e: issues.append(f"Basic functionality test failed: {e}") if issues: print("Configuration Issues:") for issue in issues: print(f" ❌ {issue}") return False else: print("✓ Configuration validated successfully") return True # Run validation validate_config()` See Also -------- * **[Installation Guide](https://edgartools.readthedocs.io/en/stable/installation/) ** - Getting started with EdgarTools * **[Quick Start](https://edgartools.readthedocs.io/en/stable/quickstart/) ** - Your first analysis * **[Performance Best Practices](https://edgartools.readthedocs.io/en/stable/resources/performance/) ** - Optimization tips * **[Troubleshooting](https://edgartools.readthedocs.io/en/stable/resources/troubleshooting/) ** - Common issues and solutions Back to top --- # Configuration - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/configuration/#configuration) Configuration ============= EdgarTools provides extensive configuration options through environment variables and programmatic settings to customize behavior, optimize performance, and ensure SEC compliance. Quick Setup ----------- For most users, you only need to set your identity: `export EDGAR_IDENTITY="Your Name your.email@company.com"` Or in Python: `from edgar import set_identity set_identity("Your Name your.email@company.com")` Environment Variables --------------------- ### Required Configuration #### EDGAR\_IDENTITY **Required for all SEC requests** Sets the User-Agent string for SEC EDGAR requests. Required by SEC to identify your application. `export EDGAR_IDENTITY="John Doe john.doe@company.com"` **Format Options:** - `"Name email@domain.com"` - Full name and email (recommended) - `"email@domain.com"` - Email only (acceptable) - `"Company Name contact@company.com"` - Company identification **Python Alternative:** `from edgar import set_identity set_identity("John Doe john.doe@company.com")` ### Performance and Access Control #### EDGAR\_ACCESS\_MODE Controls HTTP request behavior and connection limits to manage SEC server load. `export EDGAR_ACCESS_MODE="NORMAL"` **Available Modes:** | Mode | Timeout | Max Connections | Retries | Use Case | | --- | --- | --- | --- | --- | | `NORMAL` | 15s | 10 | 3 | Default - balanced performance | | `CAUTION` | 20s | 5 | 3 | Conservative - reduces server load | | `CRAWL` | 25s | 2 | 2 | Minimal impact - bulk processing | **Examples:** `# High-performance research (default) export EDGAR_ACCESS_MODE="NORMAL" # Conservative access for production export EDGAR_ACCESS_MODE="CAUTION" # Bulk data processing with minimal server impact export EDGAR_ACCESS_MODE="CRAWL"` ### Local Data Storage #### EDGAR\_USE\_LOCAL\_DATA Enables local caching of SEC data for improved performance and reduced API calls. `export EDGAR_USE_LOCAL_DATA="True"` **Values:** - `"True"`, `"true"`, `"1"` - Enable local storage - `"False"`, `"false"`, `"0"` - Disable local storage (default) **Benefits of Local Storage:** - Faster repeated access to same data - Reduced SEC API calls - Offline access to cached data - Better performance for bulk operations **Python Alternative:** `from edgar import use_local_storage use_local_storage(True)` #### EDGAR\_LOCAL\_DATA\_DIR Sets the directory for local data storage. `export EDGAR_LOCAL_DATA_DIR="/path/to/your/edgar/data"` **Default:** `~/.edgar` (user's home directory) **Directory Structure:** `~/.edgar/ # Root data directory ├── requestcache/ # HTTP response cache ├── filings/ # Downloaded filing data │ └── YYYYMMDD/ # Organized by date ├── submissions/ # Company submissions data ├── companyfacts/ # Company facts data └── reference/ # Reference data (tickers, etc.)` **Example Setup:** `# Custom data directory for project export EDGAR_LOCAL_DATA_DIR="/project/edgar_data" export EDGAR_USE_LOCAL_DATA="True"` ### Security and SSL #### EDGAR\_VERIFY\_SSL Controls SSL certificate verification for HTTPS requests. `export EDGAR_VERIFY_SSL="true"` **Values:** - `"true"` (default) - Verify SSL certificates (recommended) - `"false"`, `"0"`, `"no"`, `"n"`, `"off"` - Disable SSL verification **⚠️ Security Warning:** Only disable SSL verification in controlled environments. This reduces security by allowing man-in-the-middle attacks. **Use Cases for Disabling:** - Corporate proxy environments with custom certificates - Development environments with self-signed certificates - Network environments with SSL inspection ### HTTP Rate Limiting Rate limiting is implemented in `httpclient_ratelimiting`. The default rate limit is 9 requests per second. SEC has a maximum of 10 requests per second. To change the rate limit, call: `httpclient.update_rate_limiter(requests_per_second: int)`. ### Advanced: Distributed Rate Limiting Distributed Rate Limiting: rate limiting is implemented using [pyrate\_limiter](https://pypi.org/project/pyrate-limiter/) . To use a distributed rate limiter, such as for multiprocessing, define an httpclient.\_RATE\_LIMITER. See the pyrate\_limiter documentation and examples for details. Enterprise Configuration ------------------------ EdgarTools v4.28.0+ supports enterprise-grade configuration for custom SEC data sources and flexible rate limiting, enabling deployment with private mirrors, academic institutions, and high-volume applications. ### Configurable Rate Limiting #### EDGAR\_RATE\_LIMIT\_PER\_SEC Control the maximum number of requests per second to SEC servers or custom mirrors. `export EDGAR_RATE_LIMIT_PER_SEC="9"` **Default:** `9` requests/second (SEC's official limit) **Typical Values:** - `9` - SEC's standard limit (default, recommended for official SEC servers) - `10` - SEC's maximum allowed limit - Higher values (e.g., `20`, `50`) - Only for authorized custom mirrors with relaxed rate restrictions **Use Cases:** - **Custom mirrors**: Higher rate limits for private infrastructure with different restrictions - **Authorized applications**: High-volume applications with special SEC authorization - **Testing environments**: Flexible rate limits for development and testing - **International mirrors**: Optimized rates for regional mirrors **Example:** `# Standard SEC access (default) export EDGAR_RATE_LIMIT_PER_SEC="9" # Custom mirror with relaxed limits export EDGAR_RATE_LIMIT_PER_SEC="50" export EDGAR_BASE_URL="https://sec-mirror.company.com"` **Python Alternative:** `from edgar import httpclient # Update rate limiter programmatically httpclient.update_rate_limiter(requests_per_second=20)` **⚠️ Important:** Only use rate limits higher than 10 req/sec with custom mirrors or when authorized by SEC. Exceeding SEC's 10 req/sec limit may result in IP blocking. ### Custom SEC Data Sources Configure EdgarTools to use custom SEC mirrors, private data sources, or testing servers instead of the official SEC website. #### EDGAR\_BASE\_URL Sets the base URL for SEC EDGAR website access. `export EDGAR_BASE_URL="https://www.sec.gov"` **Default:** `https://www.sec.gov` **Use Cases:** - Corporate SEC mirrors for compliance workflows - Academic research institutions with local mirrors - Regional mirrors for reduced latency (international users) - Testing environments with mock servers **Example:** `# Corporate mirror export EDGAR_BASE_URL="https://sec-mirror.company.com" # Academic institution mirror export EDGAR_BASE_URL="https://sec.university.edu" # Regional mirror (example) export EDGAR_BASE_URL="https://sec-eu.example.com"` #### EDGAR\_DATA\_URL Sets the base URL for SEC data archives (filing documents, submissions, company facts). `export EDGAR_DATA_URL="https://data.sec.gov"` **Default:** `https://data.sec.gov` **Use Cases:** - Separate data server from website server - CDN acceleration for filing downloads - Private data repositories - Bandwidth optimization **Example:** `# Use CDN for data downloads export EDGAR_DATA_URL="https://cdn.sec-data.company.com" # Corporate data repository export EDGAR_DATA_URL="https://sec-data.company.com"` #### EDGAR\_XBRL\_URL Sets the base URL for XBRL-specific data and services. `export EDGAR_XBRL_URL="https://www.sec.gov"` **Default:** `https://www.sec.gov` **Use Cases:** - Specialized XBRL processing servers - XBRL validation and parsing services - Enhanced XBRL data repositories **Example:** `# Dedicated XBRL server export EDGAR_XBRL_URL="https://xbrl.sec-mirror.company.com"` ### Complete Enterprise Configuration Example #### Corporate Mirror Setup `# Corporate SEC mirror with higher rate limits export EDGAR_IDENTITY="Corporate Compliance compliance@company.com" export EDGAR_BASE_URL="https://sec-mirror.company.com" export EDGAR_DATA_URL="https://sec-data.company.com" export EDGAR_XBRL_URL="https://sec-xbrl.company.com" export EDGAR_RATE_LIMIT_PER_SEC="50" export EDGAR_ACCESS_MODE="NORMAL" export EDGAR_USE_LOCAL_DATA="True" export EDGAR_LOCAL_DATA_DIR="/var/lib/edgar"` #### Academic Research Institution `# University research mirror with custom rate limits export EDGAR_IDENTITY="Research Lab research@university.edu" export EDGAR_BASE_URL="https://sec.university.edu" export EDGAR_DATA_URL="https://sec-data.university.edu" export EDGAR_RATE_LIMIT_PER_SEC="25" export EDGAR_USE_LOCAL_DATA="True" export EDGAR_LOCAL_DATA_DIR="/research/edgar_data"` #### Regional Mirror (International Users) `# Regional mirror for reduced latency export EDGAR_IDENTITY="International Analyst analyst@company.com" export EDGAR_BASE_URL="https://sec-eu.example.com" export EDGAR_DATA_URL="https://sec-data-eu.example.com" export EDGAR_RATE_LIMIT_PER_SEC="15" export EDGAR_ACCESS_MODE="NORMAL"` #### Development/Testing Environment `# Mock SEC server for testing export EDGAR_IDENTITY="Developer dev@company.com" export EDGAR_BASE_URL="http://localhost:8080" export EDGAR_DATA_URL="http://localhost:8080/data" export EDGAR_XBRL_URL="http://localhost:8080/xbrl" export EDGAR_RATE_LIMIT_PER_SEC="100" # No limits for testing export EDGAR_VERIFY_SSL="false" # Self-signed certificates in dev export EDGAR_USE_RICH_LOGGING="1"` ### Python Configuration API Configure enterprise settings programmatically: `import os # Set custom SEC mirror os.environ['EDGAR_BASE_URL'] = "https://sec-mirror.company.com" os.environ['EDGAR_DATA_URL'] = "https://sec-data.company.com" os.environ['EDGAR_RATE_LIMIT_PER_SEC'] = "50" # Now import and use EdgarTools from edgar import Company company = Company("AAPL") # Uses custom mirror filings = company.get_filings(form="10-K")` **Note:** Environment variables must be set before importing EdgarTools modules, as configuration is evaluated at import time. ### Docker/Container Configuration For containerized deployments with custom SEC mirrors: `# Dockerfile FROM python:3.11-slim # Install EdgarTools RUN pip install edgartools # Configure enterprise SEC access ENV EDGAR_IDENTITY="Container App app@company.com" ENV EDGAR_BASE_URL="https://sec-mirror.company.com" ENV EDGAR_DATA_URL="https://sec-data.company.com" ENV EDGAR_RATE_LIMIT_PER_SEC="50" ENV EDGAR_ACCESS_MODE="CAUTION" ENV EDGAR_USE_LOCAL_DATA="True" ENV EDGAR_LOCAL_DATA_DIR="/app/edgar_data" # Create data directory RUN mkdir -p /app/edgar_data VOLUME /app/edgar_data WORKDIR /app` **Docker Compose Example:** `version: '3.8' services: edgar-app: image: your-edgar-app:latest environment: - EDGAR_IDENTITY=Service app@company.com - EDGAR_BASE_URL=https://sec-mirror.company.com - EDGAR_DATA_URL=https://sec-data.company.com - EDGAR_RATE_LIMIT_PER_SEC=50 - EDGAR_USE_LOCAL_DATA=True - EDGAR_LOCAL_DATA_DIR=/data volumes: - edgar-data:/data volumes: edgar-data:` ### Validation and Testing Verify your enterprise configuration: `import os from edgar import Company def validate_enterprise_config(): """Validate enterprise EdgarTools configuration.""" print("Enterprise Configuration:") print(f" Base URL: {os.getenv('EDGAR_BASE_URL', 'https://www.sec.gov')}") print(f" Data URL: {os.getenv('EDGAR_DATA_URL', 'https://data.sec.gov')}") print(f" XBRL URL: {os.getenv('EDGAR_XBRL_URL', 'https://www.sec.gov')}") print(f" Rate Limit: {os.getenv('EDGAR_RATE_LIMIT_PER_SEC', '9')} req/sec") # Test basic functionality try: company = Company("AAPL") print(f"\n✓ Successfully connected: {company.name}") # Test filing access filings = company.get_filings(form="10-K").head(1) if filings: print(f"✓ Successfully retrieved filings from: {filings[0].accession_number}") return True except Exception as e: print(f"\n❌ Configuration test failed: {e}") return False # Run validation validate_enterprise_config()` ### Troubleshooting Enterprise Configuration #### Custom Mirror Connection Issues `# Test connectivity to custom mirror import requests base_url = os.getenv('EDGAR_BASE_URL') try: response = requests.get(f"{base_url}/cgi-bin/browse-edgar") print(f"✓ Mirror accessible: {response.status_code}") except Exception as e: print(f"❌ Mirror connection failed: {e}")` #### Rate Limit Verification `# Verify rate limiter is using correct setting from edgar import httpclient print(f"Current rate limit: {os.getenv('EDGAR_RATE_LIMIT_PER_SEC', '9')} req/sec") # Monitor rate limiting in action import time start = time.time() for i in range(20): # Make 20 requests company = Company("AAPL") elapsed = time.time() - start actual_rate = 20 / elapsed print(f"Actual request rate: {actual_rate:.2f} req/sec")` #### SSL Certificate Issues with Custom Mirrors `# If custom mirror uses self-signed certificates export EDGAR_VERIFY_SSL="false" # Or configure SSL certificate bundle export REQUESTS_CA_BUNDLE="/path/to/company-ca-bundle.crt"` ### Best Practices for Enterprise Deployment 1. **Always set EDGAR\_IDENTITY** - Include company/team identification 2. **Test mirror connectivity** - Validate URLs before production deployment 3. **Monitor rate limits** - Ensure compliance with mirror's rate restrictions 4. **Use local data storage** - Enable caching for improved performance 5. **Secure credentials** - Use environment variables, not hardcoded values 6. **Document configuration** - Maintain configuration profiles for different environments 7. **Version control** - Use `.env.example` files to document required variables 8. **Health checks** - Implement validation functions to verify configuration ### Environment Variables Summary | Variable | Default | Purpose | Enterprise Use Case | | --- | --- | --- | --- | | `EDGAR_RATE_LIMIT_PER_SEC` | `9` | Request rate limit | Custom mirrors, authorized high-volume apps | | `EDGAR_BASE_URL` | `https://www.sec.gov` | SEC website base URL | Corporate mirrors, regional mirrors | | `EDGAR_DATA_URL` | `https://data.sec.gov` | Data archives URL | CDN acceleration, private repositories | | `EDGAR_XBRL_URL` | `https://www.sec.gov` | XBRL services URL | Specialized XBRL servers | ### Backward Compatibility All enterprise configuration features are **fully backward compatible**: - Default values point to official SEC servers - Zero configuration needed for standard users - Existing code continues to work without changes - Environment variables are optional ### HTTP Caching Web requests are cached by default, according to the rules defined in httpclient\_cache. #### Cache Directory The cache directory is set in `httpclient.CACHE_DIRECTORY`, set to `_cache` by default. Set CACHE\_DIRECTORY=None to disable cache. Call `httpclient.close_client()` after any changes to the CACHE\_DIRECTORY variable. #### Caching Rules The SEC marks all requests as either NO-STORE or NO-CACHE, therefore a custom cache controller was implemented with the following rules: - `/submissions` URLs for up to 10 minutes by default, set in `MAX_SUBMISSIONS_AGE_SECONDS` - `.*index/.*` URLs for up to 30 minutes by default, set in `MAX_INDEX_AGE_SECONDS` - `/Archives/edgar/data` URLs indefinitely (forever) See `httpclient_cache` for implementation. #### Advanced: Alternative Storage Caches * The underlying cache uses FileCache for local file storage. Alternative storage backends may be available through httpxthrottlecache configuration. See https://github.com/paultiq/httpxthrottlecache for details. #### EDGAR\_USE\_RICH\_LOGGING Enables enhanced console logging with rich formatting. `export EDGAR_USE_RICH_LOGGING="1"` **Values:** - `"1"` - Enable rich logging with colors and formatting - `"0"` (default) - Standard logging **Benefits:** - Color-coded log levels - Enhanced readability - Progress bars and status indicators - Better debugging information Programmatic Configuration -------------------------- ### Setting Identity `from edgar import set_identity # Set identity programmatically set_identity("Research Team research@university.edu") # Verify identity is set from edgar.core import get_identity print(f"Current identity: {get_identity()}")` ### Local Storage Control `from edgar import use_local_storage # Enable local storage use_local_storage(True) # Disable local storage use_local_storage(False) # Check current setting from edgar.storage import using_local_storage print(f"Using local storage: {using_local_storage()}")` ### HTTP Client Configuration `from edgar.core import EdgarSettings # Custom access mode custom_settings = EdgarSettings( http_timeout=30, # 30 second timeout max_connections=3, # Maximum 3 concurrent connections retries=5 # 5 retry attempts ) # Apply custom settings (requires restarting client)` Configuration Profiles ---------------------- ### Research Profile Optimized for interactive research and analysis: `export EDGAR_IDENTITY="Researcher researcher@university.edu" export EDGAR_ACCESS_MODE="NORMAL" export EDGAR_USE_LOCAL_DATA="True" export EDGAR_USE_RICH_LOGGING="1"` ### Production Profile Conservative settings for production environments: `export EDGAR_IDENTITY="Production System api@company.com" export EDGAR_ACCESS_MODE="CAUTION" export EDGAR_USE_LOCAL_DATA="True" export EDGAR_LOCAL_DATA_DIR="/var/lib/edgar" export EDGAR_VERIFY_SSL="true"` ### Bulk Processing Profile Minimal server impact for large-scale data processing: `export EDGAR_IDENTITY="Bulk Processor batch@company.com" export EDGAR_ACCESS_MODE="CRAWL" export EDGAR_USE_LOCAL_DATA="True" export EDGAR_LOCAL_DATA_DIR="/data/edgar"` ### Development Profile Flexible settings for development and testing: `export EDGAR_IDENTITY="Developer dev@company.com" export EDGAR_ACCESS_MODE="NORMAL" export EDGAR_USE_LOCAL_DATA="True" export EDGAR_USE_RICH_LOGGING="1" export EDGAR_VERIFY_SSL="false" # Only if needed for proxy` ### Enterprise Mirror Profile For custom SEC mirrors with higher rate limits (see [Enterprise Configuration](https://edgartools.readthedocs.io/en/latest/configuration/#enterprise-configuration) ): `export EDGAR_IDENTITY="Corporate Compliance compliance@company.com" export EDGAR_BASE_URL="https://sec-mirror.company.com" export EDGAR_DATA_URL="https://sec-data.company.com" export EDGAR_RATE_LIMIT_PER_SEC="50" export EDGAR_USE_LOCAL_DATA="True" export EDGAR_LOCAL_DATA_DIR="/var/lib/edgar"` Configuration File Setup ------------------------ ### .env File Create a `.env` file in your project root: `# .env file EDGAR_IDENTITY=John Doe john.doe@company.com EDGAR_ACCESS_MODE=NORMAL EDGAR_USE_LOCAL_DATA=True EDGAR_LOCAL_DATA_DIR=./edgar_data EDGAR_USE_RICH_LOGGING=1` Load with python-dotenv: `from dotenv import load_dotenv load_dotenv() # Now EdgarTools will use the environment variables from edgar import Company company = Company("AAPL")` ### Shell Configuration Add to your shell profile (`.bashrc`, `.zshrc`, etc.): `# Edgar Tools Configuration export EDGAR_IDENTITY="Your Name your.email@company.com" export EDGAR_ACCESS_MODE="NORMAL" export EDGAR_USE_LOCAL_DATA="True" export EDGAR_LOCAL_DATA_DIR="$HOME/.edgar"` Data Management --------------- ### Local Storage Benefits When `EDGAR_USE_LOCAL_DATA="True"`: 1. **Caching**: HTTP responses cached locally 2. **Offline Access**: Previously accessed data available offline 3. **Performance**: Faster subsequent access to same data 4. **Reduced API Calls**: Less load on SEC servers ### Storage Space Considerations Typical storage usage: - **Company submissions**: ~100MB for major companies - **Company facts**: ~50MB for major companies \- **HTTP cache**: Varies based on usage - **Individual filings**: 1-10MB each Troubleshooting Configuration ----------------------------- ### Check Current Configuration `import os from edgar.core import get_identity # Check identity print(f"Identity: {get_identity()}") # Check access mode print(f"Access Mode: {os.getenv('EDGAR_ACCESS_MODE', 'NORMAL')}") # Check local data settings print(f"Use Local Data: {os.getenv('EDGAR_USE_LOCAL_DATA', 'False')}") print(f"Data Directory: {os.getenv('EDGAR_LOCAL_DATA_DIR', '~/.edgar')}") # Check SSL verification print(f"Verify SSL: {os.getenv('EDGAR_VERIFY_SSL', 'true')}")` ### Common Issues #### Identity Not Set `# Error: No identity set # Solution: set_identity("Your Name your.email@company.com")` #### Permission Errors `# Error: Permission denied writing to ~/.edgar # Solution: Check directory permissions or use custom directory export EDGAR_LOCAL_DATA_DIR="/tmp/edgar"` #### SSL Verification Errors `# Error: SSL certificate verification failed # Solution: Disable SSL verification (only if safe) export EDGAR_VERIFY_SSL="false"` #### Connection Timeouts `# Error: Connection timeouts in slow network # Solution: Use more conservative settings export EDGAR_ACCESS_MODE="CAUTION"` Security Best Practices ----------------------- 1. **Always set EDGAR\_IDENTITY** - Required for SEC compliance 2. **Keep SSL verification enabled** - Only disable in controlled environments 3. **Secure data directory** - Ensure appropriate file permissions 4. **Use least-privilege access** - Don't run with unnecessary elevated permissions 5. **Monitor data usage** - Be aware of local storage space consumption Docker Configuration -------------------- For containerized deployments: `# Dockerfile ENV EDGAR_IDENTITY="Container App app@company.com" ENV EDGAR_ACCESS_MODE="CAUTION" ENV EDGAR_USE_LOCAL_DATA="True" ENV EDGAR_LOCAL_DATA_DIR="/app/edgar_data" # Create data directory RUN mkdir -p /app/edgar_data VOLUME /app/edgar_data` Configuration Validation ------------------------ Validate your configuration before running analysis: `from edgar import Company import os def validate_config(): """Validate EdgarTools configuration.""" issues = [] # Check identity try: from edgar.core import get_identity identity = get_identity() if not identity: issues.append("EDGAR_IDENTITY not set") elif "@" not in identity: issues.append("EDGAR_IDENTITY should include email") except: issues.append("Cannot retrieve EDGAR_IDENTITY") # Check data directory if os.getenv('EDGAR_USE_LOCAL_DATA', 'False').lower() in ['true', '1']: data_dir = os.getenv('EDGAR_LOCAL_DATA_DIR', '~/.edgar') expanded_dir = os.path.expanduser(data_dir) if not os.path.exists(expanded_dir): try: os.makedirs(expanded_dir, exist_ok=True) except: issues.append(f"Cannot create data directory: {data_dir}") # Test basic functionality try: company = Company("AAPL") print(f"✓ Successfully created company: {company.name}") except Exception as e: issues.append(f"Basic functionality test failed: {e}") if issues: print("Configuration Issues:") for issue in issues: print(f" ❌ {issue}") return False else: print("✓ Configuration validated successfully") return True # Run validation validate_config()` See Also -------- * **[Installation Guide](https://edgartools.readthedocs.io/en/latest/installation/) ** - Getting started with EdgarTools * **[Quick Start](https://edgartools.readthedocs.io/en/latest/quickstart/) ** - Your first analysis * **[Performance Best Practices](https://edgartools.readthedocs.io/en/latest/resources/performance/) ** - Optimization tips * **[Troubleshooting](https://edgartools.readthedocs.io/en/latest/resources/troubleshooting/) ** - Common issues and solutions Back to top --- # The Complete Guide to SEC Filings in Python (2026) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/complete-guide/#the-complete-guide-to-sec-filings-in-python) The Complete Guide to SEC Filings in Python =========================================== _Last updated: February 2026_ Every public company in the United States is required to file financial reports with the Securities and Exchange Commission (SEC). These filings — annual reports (10-K), quarterly reports (10-Q), current event reports (8-K), insider trading disclosures (Form 4), institutional holdings (13F), and dozens more — are all publicly available through the SEC's EDGAR database. EDGAR contains over 20 million filings going back to 1994. It's the single richest source of corporate financial data in the world, and it's completely free. EdgarTools is a Python library that turns EDGAR's raw data into structured Python objects you can analyze immediately. No API key, no paid subscription, no rate-limited trial. Just `pip install edgartools` and you have access to financial statements, insider trades, institutional holdings, fund portfolios, proxy statements, and more — all as native Python objects with built-in analysis methods. `from edgar import Company income = (Company("AAPL") .get_filings(form="10-K")[0].obj() .financials .income_statement() ) print(income)` That's a complete Apple income statement in three lines. This guide shows you everything else you can do. * * * Table of Contents ----------------- * [Installation](https://edgartools.readthedocs.io/en/latest/complete-guide/#installation) * [Finding Companies](https://edgartools.readthedocs.io/en/latest/complete-guide/#finding-companies) * [Working with Filings](https://edgartools.readthedocs.io/en/latest/complete-guide/#working-with-filings) * [Financial Statements](https://edgartools.readthedocs.io/en/latest/complete-guide/#financial-statements) * [Company Facts](https://edgartools.readthedocs.io/en/latest/complete-guide/#company-facts) * [Insider Trading](https://edgartools.readthedocs.io/en/latest/complete-guide/#insider-trading) * [Institutional Holdings (13F)](https://edgartools.readthedocs.io/en/latest/complete-guide/#institutional-holdings-13f) * [Investment Funds](https://edgartools.readthedocs.io/en/latest/complete-guide/#investment-funds) * [Current Filings](https://edgartools.readthedocs.io/en/latest/complete-guide/#current-filings) * [AI and MCP Integration](https://edgartools.readthedocs.io/en/latest/complete-guide/#ai-and-mcp-integration) * [EdgarTools vs Alternatives](https://edgartools.readthedocs.io/en/latest/complete-guide/#edgartools-vs-alternatives) * [Resources](https://edgartools.readthedocs.io/en/latest/complete-guide/#resources) * * * Installation ------------ `pip install edgartools` The SEC requires all API users to identify themselves. Set your identity once per session: `from edgar import set_identity set_identity("Your Name your.email@example.com")` You can also set the `EDGAR_IDENTITY` environment variable so you don't need to call this in every script. Wrong package? If you see `ImportError: cannot import name 'get_filings' from 'edgar'`, you may have the wrong `edgar` package installed: `pip uninstall edgar && pip install edgartools` [Open in Colab](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/01_getting_started.ipynb) * * * Finding Companies ----------------- Look up any public company by ticker symbol, CIK number, or name: `from edgar import Company apple = Company("AAPL") # By ticker microsoft = Company("MSFT") # Another ticker berkshire = Company(1067983) # By CIK number` The `Company` object gives you access to metadata, filings, financials, and facts — all from a single entry point: `company = Company("AAPL") company.name # 'APPLE INC' company.cik # 320193 company.sic_code # '3571' company.sic_description # 'Electronic Computers' company.shares_outstanding # 15115785000.0 company.public_float # 2899948348000.0` From a `Company` object you can get filings, financials, facts, and more — it's the starting point for most analysis workflows in edgartools. [Open in Colab](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-company-data-python.ipynb) * * * Working with Filings -------------------- ### Get a Company's Filings Every company's full filing history is available: `filings = Company("MSFT").get_filings() # All filings tenks = Company("MSFT").get_filings(form="10-K") # Just 10-Ks` ### Filter and Search Narrow results by date, form type, or other criteria: `filings = Company("TSLA").get_filings(form="10-K") recent = filings.head(5) # Most recent 5 filing = filings[0] # Latest filing` ### Open and Read Filings Once you have a filing, you can open it in your browser, extract clean text for NLP, or convert it to markdown: `filing = Company("AAPL").get_filings(form="10-K")[0] filing.open() # Open in your browser text = filing.text() # Get clean text content md = filing.markdown() # Get markdown version html = filing.html() # Get raw HTML` The `text()` method strips all HTML formatting and returns clean, readable text — useful for NLP pipelines, RAG systems, and text analysis. ### Typed Data Objects The real power of edgartools is `filing.obj()` — it converts raw filings into structured Python objects with properties, methods, and DataFrames: `tenk = filing.obj() # Returns a TenK object # Access sections tenk.business_description tenk.risk_factors tenk.mda # Management Discussion & Analysis # Access financials tenk.financials.income_statement() tenk.financials.balance_sheet() tenk.financials.cash_flow_statement() # Auditor and corporate structure tenk.auditor # AuditorInfo (name, PCAOB ID, location) tenk.subsidiaries # SubsidiaryList from Exhibit 21 # XBRL report pages (statements, notes, details) tenk.reports # Reports from FilingSummary.xml` EdgarTools provides typed data objects for 17+ filing types including 10-K, 10-Q, 8-K, 13F, Form 4, DEF 14A, N-PORT, N-MFP, Form D, Form C, and more. See the full list in the [Filing Types](https://edgartools.readthedocs.io/en/latest/data-objects/) documentation. [Open in Colab](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/download-10k-annual-report-python.ipynb) * * * Financial Statements -------------------- ### From a Single Filing Extract income statements, balance sheets, and cash flow statements from any 10-K or 10-Q: `from edgar import Company tenk = Company("AAPL").get_filings(form="10-K")[0].obj() income = tenk.financials.income_statement() balance = tenk.financials.balance_sheet() cashflow = tenk.financials.cash_flow_statement()` ### Convert to DataFrames Every statement converts to a pandas DataFrame for analysis: `df = income.to_dataframe()` ### Multi-Period Analysis Get financial data across multiple years using the XBRL stitching API: `from edgar import Company company = Company("MSFT") financials = company.get_financials() # Multi-year income statement income = financials.income_statement() # Multi-year balance sheet balance = financials.balance_sheet()` [Open in Colab](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/financial-statements-sec-python.ipynb)   [Multi-Period Analysis](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/XBRL2-StitchingStatements.ipynb) See it live on edgar.tools The code above extracts financials programmatically. **[edgar.tools](https://app.edgar.tools/?utm_source=edgartools-docs&utm_medium=see-live&utm_content=complete-guide) ** renders the same statements visually — multi-year income, balance sheet, and cash flow for any company, with export to Excel, PDF, or CSV. * **[See Apple's multi-year financials →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=complete-guide) ** * **[See Microsoft's financials →](https://app.edgar.tools/companies/MSFT?utm_source=edgartools-docs&utm_medium=see-live&utm_content=complete-guide) ** * **[Browse 12 XBRL disclosure topics →](https://app.edgar.tools/disclosures?utm_source=edgartools-docs&utm_medium=see-live&utm_content=complete-guide) ** Also includes insider trades, 13F holdings, real-time filing stream, REST API, and hosted MCP server. Free tier available. * * * Company Facts ------------- Track individual financial metrics across a company's entire filing history using the SEC's XBRL facts database: `from edgar import Company facts = Company("GOOG").get_facts() # Get specific metrics facts.get_revenue() # Latest annual revenue facts.get_net_income() # Latest net income facts.get_total_assets() # Latest total assets facts.get_shareholders_equity() # Latest equity` Query any XBRL concept: `# Get a specific concept's value facts.get_concept("AccountsPayableCurrent") # Time series for any metric facts.time_series("Revenues")` [Open in Colab](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/extract-revenue-earnings-python.ipynb) * * * Insider Trading --------------- Track insider buying and selling through SEC Form 4 filings: `from edgar import Company form4s = Company("TSLA").get_filings(form="4").head(5) for f in form4s: ownership = f.obj() print(ownership)` Each Form 4 filing is parsed into an `Ownership` object with structured transaction details — who traded, how many shares, at what price, and whether it was a buy or sell. Convert transactions to a DataFrame for analysis across multiple filings: `import pandas as pd form4s = Company("NVDA").get_filings(form="4").head(20) transactions = pd.concat([f.obj().to_dataframe().fillna('') for f in form4s])` This gives you a single DataFrame of all insider transactions, ready for filtering by insider name, transaction type, or date range. [Open in Colab](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/insider-trading-sec-form4-python.ipynb) * * * Institutional Holdings (13F) ---------------------------- See what hedge funds and institutional investors are holding. Every institutional manager with $100M+ in assets must file quarterly 13F reports: `from edgar import Company # Citadel Advisors' latest 13F thirteenf = Company(1423053).get_filings(form="13F-HR")[0].obj() print(thirteenf.holdings) # Full portfolio as DataFrame` Compare holdings quarter-over-quarter: `thirteenf.compare_holdings() # NEW, CLOSED, INCREASED, DECREASED thirteenf.holding_history(periods=4) # Multi-quarter trends` [Open in Colab](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/13f-institutional-holdings-python.ipynb) * * * Investment Funds ---------------- ### Mutual Fund & ETF Holdings (N-PORT) Analyze complete portfolio holdings from monthly N-PORT filings: `from edgar import Company fund = Company("VANGUARD INDEX FUNDS") nport = fund.get_filings(form="NPORT-P")[0].obj() nport.investments # Full holdings` ### Money Market Funds (N-MFP) `mmf = fund.get_filings(form="N-MFP2")[0].obj()` ### Executive Compensation and Proxy Statements Parse DEF 14A proxy statements for executive pay, board composition, and shareholder proposals: `proxy = Company("AAPL").get_filings(form="DEF 14A")[0].obj() proxy.peo_name # CEO name proxy.peo_total_comp # CEO total compensation proxy.executive_compensation # Multi-year compensation DataFrame` [Executive Compensation](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/executive-compensation-sec-python.ipynb) ### Business Development Companies Analyze BDC portfolio investments and lending activity: `bdc = Company("ARCC").get_filings(form="10-K")[0].obj()` [N-PORT](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/mutual-fund-holdings-nport-python.ipynb)   [N-MFP](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/money-market-fund-nmfp-python.ipynb)   [BDC](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/bdc-business-development-company-python.ipynb) * * * Current Filings --------------- Monitor SEC filings as they're published. The SEC updates its filing index throughout the day, and edgartools gives you access to the latest submissions: `from edgar import get_current_filings filings = get_current_filings() # Everything filed today eightks = filings.filter(form="8-K") # Just current events tenks = filings.filter(form="10-K") # Just annual reports` This is useful for building filing alert systems, monitoring specific companies for new submissions, or tracking daily filing activity across the market. [Open in Colab](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-filings-today-python.ipynb) * * * AI and MCP Integration ---------------------- EdgarTools includes a built-in MCP (Model Context Protocol) server, enabling AI assistants like Claude to query SEC data directly through natural language: `# Run the MCP server edgartools-mcp # Or with uvx (no install needed) uvx edgartools-mcp` Once connected, you can ask an AI assistant questions like "What was Apple's revenue last year?" or "Show me Elon Musk's recent stock sales" and it will use edgartools to fetch and analyze the data. The MCP server exposes tools for company research, filing search, financial analysis, ownership tracking, and multi-company comparison — giving AI agents structured access to the full SEC EDGAR database. This works with Claude Desktop, Claude Code, and any MCP-compatible AI client. See the [AI Integration Guide](https://edgartools.readthedocs.io/en/latest/ai-integration/) for setup instructions. * * * EdgarTools vs Alternatives -------------------------- ### EdgarTools vs sec-api.io | Feature | EdgarTools | sec-api.io | | --- | --- | --- | | **Price** | Free | $49–$239/month | | **API key required** | No | Yes | | **Open source** | Yes (MIT) | No | | **Lines of code for financials** | 3 | 15+ | | **XBRL parsing** | Native Python objects | JSON | | **Filing types with typed objects** | 17+ | Raw download | | **Works offline (with cache)** | Yes | No | | **AI/MCP integration** | Built-in | No | ### EdgarTools vs sec-edgar-downloader `sec-edgar-downloader` downloads raw filing documents to disk. It's a downloader — you get HTML and XML files, then you're on your own for parsing. EdgarTools downloads _and_ parses filings into structured Python objects with properties, methods, and DataFrames. If you need to analyze the data (not just store the files), edgartools does both. ### EdgarTools vs python-edgar `python-edgar` provides basic access to the EDGAR full-text search and filing index. EdgarTools provides that plus XBRL parsing, typed data objects for 17+ filing types, financial statement extraction, rich terminal display, and Jupyter/Colab integration. ### When to Use What * **Need structured financial analysis in Python?** Use EdgarTools — it's built for this. * **Need a hosted API with real-time WebSocket streams?** sec-api.io offers infrastructure features a client library can't. * **Just need to bulk download filing documents?** sec-edgar-downloader is a simple file downloader. * **Need to work in a language other than Python?** sec-api.io offers REST endpoints for any language. * * * Common Use Cases ---------------- ### Financial Research and Modeling Pull income statements, balance sheets, and cash flows across multiple years for any public company. Build financial models, calculate ratios, and compare companies — all without leaving Python. `from edgar import Company # Get multi-year financials for modeling financials = Company("AMZN").get_financials() income = financials.income_statement() df = income.to_dataframe() # Ready for pandas analysis` ### NLP and Text Analysis Extract clean text from filings for sentiment analysis, topic modeling, or training language models on financial documents: `filing = Company("JPM").get_filings(form="10-K")[0] text = filing.text() # Clean text, no HTML` ### Portfolio Monitoring Track what institutional investors are buying and selling each quarter, monitor insider transactions for signal detection, and watch for material events through 8-K filings. ### Academic Research EdgarTools is used in academic settings for corporate governance studies, market efficiency research, and large-scale financial data analysis. The structured data objects and DataFrame outputs integrate directly into scientific Python workflows (pandas, numpy, scikit-learn). ### Building RAG Systems The `text()` and `markdown()` methods produce clean document representations suitable for chunking and embedding in retrieval-augmented generation (RAG) pipelines. Combined with the MCP server, edgartools can serve as a live data source for AI-powered financial research assistants and chatbots. * * * Resources --------- ### Documentation * [Quick Start](https://edgartools.readthedocs.io/en/latest/quickstart/) — Your first analysis in 5 minutes * [Financial Data Guide](https://edgartools.readthedocs.io/en/latest/guides/financial-data/) — Income statements, balance sheets, cash flow * [Filing Types](https://edgartools.readthedocs.io/en/latest/data-objects/) — All 17+ supported filing types * [XBRL Deep Dive](https://edgartools.readthedocs.io/en/latest/xbrl/) — Advanced structured financial data * [API Reference](https://edgartools.readthedocs.io/en/latest/api/company/) — Complete class documentation ### Interactive Notebooks 54 Colab-ready notebooks covering every feature — browse the full collection on the [Notebooks](https://edgartools.readthedocs.io/en/latest/notebooks/) page. ### Community * [GitHub](https://github.com/dgunning/edgartools) — Source code, issues, and discussions * [PyPI](https://pypi.org/project/edgartools/) — Package releases * [ReadTheDocs](https://edgartools.readthedocs.io/) — Full documentation * * * _EdgarTools is free and open source (MIT license). No API key, no subscription, no limits. Just `pip install edgartools` and start analyzing SEC data. Built and maintained by the open source community._ Back to top --- # The Complete Guide to SEC Filings in Python (2026) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/complete-guide/#the-complete-guide-to-sec-filings-in-python) The Complete Guide to SEC Filings in Python =========================================== _Last updated: February 2026_ Every public company in the United States is required to file financial reports with the Securities and Exchange Commission (SEC). These filings — annual reports (10-K), quarterly reports (10-Q), current event reports (8-K), insider trading disclosures (Form 4), institutional holdings (13F), and dozens more — are all publicly available through the SEC's EDGAR database. EDGAR contains over 20 million filings going back to 1994. It's the single richest source of corporate financial data in the world, and it's completely free. EdgarTools is a Python library that turns EDGAR's raw data into structured Python objects you can analyze immediately. No API key, no paid subscription, no rate-limited trial. Just `pip install edgartools` and you have access to financial statements, insider trades, institutional holdings, fund portfolios, proxy statements, and more — all as native Python objects with built-in analysis methods. `from edgar import Company income = (Company("AAPL") .get_filings(form="10-K")[0].obj() .financials .income_statement() ) print(income)` That's a complete Apple income statement in three lines. This guide shows you everything else you can do. * * * Table of Contents ----------------- * [Installation](https://edgartools.readthedocs.io/en/stable/complete-guide/#installation) * [Finding Companies](https://edgartools.readthedocs.io/en/stable/complete-guide/#finding-companies) * [Working with Filings](https://edgartools.readthedocs.io/en/stable/complete-guide/#working-with-filings) * [Financial Statements](https://edgartools.readthedocs.io/en/stable/complete-guide/#financial-statements) * [Company Facts](https://edgartools.readthedocs.io/en/stable/complete-guide/#company-facts) * [Insider Trading](https://edgartools.readthedocs.io/en/stable/complete-guide/#insider-trading) * [Institutional Holdings (13F)](https://edgartools.readthedocs.io/en/stable/complete-guide/#institutional-holdings-13f) * [Investment Funds](https://edgartools.readthedocs.io/en/stable/complete-guide/#investment-funds) * [Current Filings](https://edgartools.readthedocs.io/en/stable/complete-guide/#current-filings) * [AI and MCP Integration](https://edgartools.readthedocs.io/en/stable/complete-guide/#ai-and-mcp-integration) * [EdgarTools vs Alternatives](https://edgartools.readthedocs.io/en/stable/complete-guide/#edgartools-vs-alternatives) * [Resources](https://edgartools.readthedocs.io/en/stable/complete-guide/#resources) * * * Installation ------------ `pip install edgartools` The SEC requires all API users to identify themselves. Set your identity once per session: `from edgar import set_identity set_identity("Your Name your.email@example.com")` You can also set the `EDGAR_IDENTITY` environment variable so you don't need to call this in every script. Wrong package? If you see `ImportError: cannot import name 'get_filings' from 'edgar'`, you may have the wrong `edgar` package installed: `pip uninstall edgar && pip install edgartools` [Open in Colab](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/01_getting_started.ipynb) * * * Finding Companies ----------------- Look up any public company by ticker symbol, CIK number, or name: `from edgar import Company apple = Company("AAPL") # By ticker microsoft = Company("MSFT") # Another ticker berkshire = Company(1067983) # By CIK number` The `Company` object gives you access to metadata, filings, financials, and facts — all from a single entry point: `company = Company("AAPL") company.name # 'APPLE INC' company.cik # 320193 company.sic_code # '3571' company.sic_description # 'Electronic Computers' company.shares_outstanding # 15115785000.0 company.public_float # 2899948348000.0` From a `Company` object you can get filings, financials, facts, and more — it's the starting point for most analysis workflows in edgartools. [Open in Colab](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-company-data-python.ipynb) * * * Working with Filings -------------------- ### Get a Company's Filings Every company's full filing history is available: `filings = Company("MSFT").get_filings() # All filings tenks = Company("MSFT").get_filings(form="10-K") # Just 10-Ks` ### Filter and Search Narrow results by date, form type, or other criteria: `filings = Company("TSLA").get_filings(form="10-K") recent = filings.head(5) # Most recent 5 filing = filings[0] # Latest filing` ### Open and Read Filings Once you have a filing, you can open it in your browser, extract clean text for NLP, or convert it to markdown: `filing = Company("AAPL").get_filings(form="10-K")[0] filing.open() # Open in your browser text = filing.text() # Get clean text content md = filing.markdown() # Get markdown version html = filing.html() # Get raw HTML` The `text()` method strips all HTML formatting and returns clean, readable text — useful for NLP pipelines, RAG systems, and text analysis. ### Typed Data Objects The real power of edgartools is `filing.obj()` — it converts raw filings into structured Python objects with properties, methods, and DataFrames: `tenk = filing.obj() # Returns a TenK object # Access sections tenk.business_description tenk.risk_factors tenk.mda # Management Discussion & Analysis # Access financials tenk.financials.income_statement() tenk.financials.balance_sheet() tenk.financials.cash_flow_statement() # Auditor and corporate structure tenk.auditor # AuditorInfo (name, PCAOB ID, location) tenk.subsidiaries # SubsidiaryList from Exhibit 21 # XBRL report pages (statements, notes, details) tenk.reports # Reports from FilingSummary.xml` EdgarTools provides typed data objects for 17+ filing types including 10-K, 10-Q, 8-K, 13F, Form 4, DEF 14A, N-PORT, N-MFP, Form D, Form C, and more. See the full list in the [Filing Types](https://edgartools.readthedocs.io/en/stable/data-objects/) documentation. [Open in Colab](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/download-10k-annual-report-python.ipynb) * * * Financial Statements -------------------- ### From a Single Filing Extract income statements, balance sheets, and cash flow statements from any 10-K or 10-Q: `from edgar import Company tenk = Company("AAPL").get_filings(form="10-K")[0].obj() income = tenk.financials.income_statement() balance = tenk.financials.balance_sheet() cashflow = tenk.financials.cash_flow_statement()` ### Convert to DataFrames Every statement converts to a pandas DataFrame for analysis: `df = income.to_dataframe()` ### Multi-Period Analysis Get financial data across multiple years using the XBRL stitching API: `from edgar import Company company = Company("MSFT") financials = company.get_financials() # Multi-year income statement income = financials.income_statement() # Multi-year balance sheet balance = financials.balance_sheet()` [Open in Colab](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/financial-statements-sec-python.ipynb)   [Multi-Period Analysis](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/XBRL2-StitchingStatements.ipynb) See it live on edgar.tools The code above extracts financials programmatically. **[edgar.tools](https://app.edgar.tools/?utm_source=edgartools-docs&utm_medium=see-live&utm_content=complete-guide) ** renders the same statements visually — multi-year income, balance sheet, and cash flow for any company, with export to Excel, PDF, or CSV. * **[See Apple's multi-year financials →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=complete-guide) ** * **[See Microsoft's financials →](https://app.edgar.tools/companies/MSFT?utm_source=edgartools-docs&utm_medium=see-live&utm_content=complete-guide) ** * **[Browse 12 XBRL disclosure topics →](https://app.edgar.tools/disclosures?utm_source=edgartools-docs&utm_medium=see-live&utm_content=complete-guide) ** Also includes insider trades, 13F holdings, real-time filing stream, REST API, and hosted MCP server. Free tier available. * * * Company Facts ------------- Track individual financial metrics across a company's entire filing history using the SEC's XBRL facts database: `from edgar import Company facts = Company("GOOG").get_facts() # Get specific metrics facts.get_revenue() # Latest annual revenue facts.get_net_income() # Latest net income facts.get_total_assets() # Latest total assets facts.get_shareholders_equity() # Latest equity` Query any XBRL concept: `# Get a specific concept's value facts.get_concept("AccountsPayableCurrent") # Time series for any metric facts.time_series("Revenues")` [Open in Colab](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/extract-revenue-earnings-python.ipynb) * * * Insider Trading --------------- Track insider buying and selling through SEC Form 4 filings: `from edgar import Company form4s = Company("TSLA").get_filings(form="4").head(5) for f in form4s: ownership = f.obj() print(ownership)` Each Form 4 filing is parsed into an `Ownership` object with structured transaction details — who traded, how many shares, at what price, and whether it was a buy or sell. Convert transactions to a DataFrame for analysis across multiple filings: `import pandas as pd form4s = Company("NVDA").get_filings(form="4").head(20) transactions = pd.concat([f.obj().to_dataframe().fillna('') for f in form4s])` This gives you a single DataFrame of all insider transactions, ready for filtering by insider name, transaction type, or date range. [Open in Colab](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/insider-trading-sec-form4-python.ipynb) * * * Institutional Holdings (13F) ---------------------------- See what hedge funds and institutional investors are holding. Every institutional manager with $100M+ in assets must file quarterly 13F reports: `from edgar import Company # Citadel Advisors' latest 13F thirteenf = Company(1423053).get_filings(form="13F-HR")[0].obj() print(thirteenf.holdings) # Full portfolio as DataFrame` Compare holdings quarter-over-quarter: `thirteenf.compare_holdings() # NEW, CLOSED, INCREASED, DECREASED thirteenf.holding_history(periods=4) # Multi-quarter trends` [Open in Colab](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/13f-institutional-holdings-python.ipynb) * * * Investment Funds ---------------- ### Mutual Fund & ETF Holdings (N-PORT) Analyze complete portfolio holdings from monthly N-PORT filings: `from edgar import Company fund = Company("VANGUARD INDEX FUNDS") nport = fund.get_filings(form="NPORT-P")[0].obj() nport.investments # Full holdings` ### Money Market Funds (N-MFP) `mmf = fund.get_filings(form="N-MFP2")[0].obj()` ### Executive Compensation and Proxy Statements Parse DEF 14A proxy statements for executive pay, board composition, and shareholder proposals: `proxy = Company("AAPL").get_filings(form="DEF 14A")[0].obj() proxy.peo_name # CEO name proxy.peo_total_comp # CEO total compensation proxy.executive_compensation # Multi-year compensation DataFrame` [Executive Compensation](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/executive-compensation-sec-python.ipynb) ### Business Development Companies Analyze BDC portfolio investments and lending activity: `bdc = Company("ARCC").get_filings(form="10-K")[0].obj()` [N-PORT](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/mutual-fund-holdings-nport-python.ipynb)   [N-MFP](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/money-market-fund-nmfp-python.ipynb)   [BDC](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/bdc-business-development-company-python.ipynb) * * * Current Filings --------------- Monitor SEC filings as they're published. The SEC updates its filing index throughout the day, and edgartools gives you access to the latest submissions: `from edgar import get_current_filings filings = get_current_filings() # Everything filed today eightks = filings.filter(form="8-K") # Just current events tenks = filings.filter(form="10-K") # Just annual reports` This is useful for building filing alert systems, monitoring specific companies for new submissions, or tracking daily filing activity across the market. [Open in Colab](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-filings-today-python.ipynb) * * * AI and MCP Integration ---------------------- EdgarTools includes a built-in MCP (Model Context Protocol) server, enabling AI assistants like Claude to query SEC data directly through natural language: `# Run the MCP server edgartools-mcp # Or with uvx (no install needed) uvx edgartools-mcp` Once connected, you can ask an AI assistant questions like "What was Apple's revenue last year?" or "Show me Elon Musk's recent stock sales" and it will use edgartools to fetch and analyze the data. The MCP server exposes tools for company research, filing search, financial analysis, ownership tracking, and multi-company comparison — giving AI agents structured access to the full SEC EDGAR database. This works with Claude Desktop, Claude Code, and any MCP-compatible AI client. See the [AI Integration Guide](https://edgartools.readthedocs.io/en/stable/ai-integration/) for setup instructions. * * * EdgarTools vs Alternatives -------------------------- ### EdgarTools vs sec-api.io | Feature | EdgarTools | sec-api.io | | --- | --- | --- | | **Price** | Free | $49–$239/month | | **API key required** | No | Yes | | **Open source** | Yes (MIT) | No | | **Lines of code for financials** | 3 | 15+ | | **XBRL parsing** | Native Python objects | JSON | | **Filing types with typed objects** | 17+ | Raw download | | **Works offline (with cache)** | Yes | No | | **AI/MCP integration** | Built-in | No | ### EdgarTools vs sec-edgar-downloader `sec-edgar-downloader` downloads raw filing documents to disk. It's a downloader — you get HTML and XML files, then you're on your own for parsing. EdgarTools downloads _and_ parses filings into structured Python objects with properties, methods, and DataFrames. If you need to analyze the data (not just store the files), edgartools does both. ### EdgarTools vs python-edgar `python-edgar` provides basic access to the EDGAR full-text search and filing index. EdgarTools provides that plus XBRL parsing, typed data objects for 17+ filing types, financial statement extraction, rich terminal display, and Jupyter/Colab integration. ### When to Use What * **Need structured financial analysis in Python?** Use EdgarTools — it's built for this. * **Need a hosted API with real-time WebSocket streams?** sec-api.io offers infrastructure features a client library can't. * **Just need to bulk download filing documents?** sec-edgar-downloader is a simple file downloader. * **Need to work in a language other than Python?** sec-api.io offers REST endpoints for any language. * * * Common Use Cases ---------------- ### Financial Research and Modeling Pull income statements, balance sheets, and cash flows across multiple years for any public company. Build financial models, calculate ratios, and compare companies — all without leaving Python. `from edgar import Company # Get multi-year financials for modeling financials = Company("AMZN").get_financials() income = financials.income_statement() df = income.to_dataframe() # Ready for pandas analysis` ### NLP and Text Analysis Extract clean text from filings for sentiment analysis, topic modeling, or training language models on financial documents: `filing = Company("JPM").get_filings(form="10-K")[0] text = filing.text() # Clean text, no HTML` ### Portfolio Monitoring Track what institutional investors are buying and selling each quarter, monitor insider transactions for signal detection, and watch for material events through 8-K filings. ### Academic Research EdgarTools is used in academic settings for corporate governance studies, market efficiency research, and large-scale financial data analysis. The structured data objects and DataFrame outputs integrate directly into scientific Python workflows (pandas, numpy, scikit-learn). ### Building RAG Systems The `text()` and `markdown()` methods produce clean document representations suitable for chunking and embedding in retrieval-augmented generation (RAG) pipelines. Combined with the MCP server, edgartools can serve as a live data source for AI-powered financial research assistants and chatbots. * * * Resources --------- ### Documentation * [Quick Start](https://edgartools.readthedocs.io/en/stable/quickstart/) — Your first analysis in 5 minutes * [Financial Data Guide](https://edgartools.readthedocs.io/en/stable/guides/financial-data/) — Income statements, balance sheets, cash flow * [Filing Types](https://edgartools.readthedocs.io/en/stable/data-objects/) — All 17+ supported filing types * [XBRL Deep Dive](https://edgartools.readthedocs.io/en/stable/xbrl/) — Advanced structured financial data * [API Reference](https://edgartools.readthedocs.io/en/stable/api/company/) — Complete class documentation ### Interactive Notebooks 54 Colab-ready notebooks covering every feature — browse the full collection on the [Notebooks](https://edgartools.readthedocs.io/en/stable/notebooks/) page. ### Community * [GitHub](https://github.com/dgunning/edgartools) — Source code, issues, and discussions * [PyPI](https://pypi.org/project/edgartools/) — Package releases * [ReadTheDocs](https://edgartools.readthedocs.io/) — Full documentation * * * _EdgarTools is free and open source (MIT license). No API key, no subscription, no limits. Just `pip install edgartools` and start analyzing SEC data. Built and maintained by the open source community._ Back to top --- # Track Insiders - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/company-insiders/#company-insiders-find-officers-directors-and-major-shareholders) Company Insiders: Find Officers, Directors, and Major Shareholders ================================================================== This guide shows how to get a list of insiders for a company by writing a simple script to loop through their Form 4 filings and getting the **name** and **position**. 1\. Deciding on an appropriate date range ----------------------------------------- The approach is to get all Form 4 Insider filings for the past 6 months. To specify the date range we use a use `timedelta` to subtract 6 months from `datetime.now()` `from datetime import datetime, timedelta from edgar import * date_range = ((datetime.now() - timedelta(days=6*30)) # Approximate 6 months .strftime('%Y-%m-%d:'))` 2\. Getting the company filings ------------------------------- Now we can use the `Company` class to get the company filings for the past 6 months. `c: Company = Company(ticker) filings: EntityFilings = c.get_filings(form='4', filing_date=date_range)` 3\. Collecting data from each Form 4 ------------------------------------ Now we loop through each filing and get the ownership summary, which contains the insider names and their positions. Each Form4 has an `OwnershipSummary` object that we can convert to a DataFrame. `dfs = [] # List to hold DataFrames for each filing for filing in tqdm(filings): form4: Form4 = filing.obj() summary = form4.get_ownership_summary() dfs.append(summary.to_dataframe()[['Insider', 'Position']])` 4\. Combining the DataFrames ---------------------------- Finally, we can concatenate all the DataFrames into a single DataFrame and drop duplicates to get a unique list of insiders. `import pandas as pd insiders = (pd.concat(dfs, ignore_index=True) .drop_duplicates().reset_index(drop=True) .sort_values(by='Position', key=lambda col: col == 'Director', ascending=True) )` 5\. Putting it all together --------------------------- The complete code to get the insiders for a company is as follows. Note that we put it inside a function so we can easily reuse it for different tickers. `import pandas as pd from rich import print from tqdm.auto import tqdm from edgar import * from edgar.entity import EntityFilings from edgar.ownership import Form4 from datetime import datetime, timedelta # Calculate the date 6 months ago from today date_range = ((datetime.now() - timedelta(days=6*30)) # Approximate 6 months .strftime('%Y-%m-%d:')) def get_insiders(ticker): c: Company = Company(ticker) filings: EntityFilings = c.get_filings(form='4', filing_date=date_range) dfs = [] for filing in tqdm(filings): form4: Form4 = filing.obj() summary = form4.get_ownership_summary() dfs.append(summary.to_dataframe()[['Insider', 'Position']]) insiders = (pd.concat(dfs, ignore_index=True) .drop_duplicates().reset_index(drop=True) .sort_values(by='Position', key=lambda col: col == 'Director', ascending=True) ) return insiders if __name__ == '__main__': insiders = get_insiders("NFLX") print(insiders)` See this on edgar.tools The script above loops through Form 4 filings to build an insider list for one company. **edgar.tools** has this pre-computed across 186K+ insider filings with 802K+ transactions — including net buy/sell sentiment and executive profiles. * **[See Netflix's insiders and transactions instantly →](https://app.edgar.tools/companies/NFLX?utm_source=edgartools-docs&utm_medium=see-live&utm_content=company-insiders) ** * **[See Apple's insider trading activity →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=company-insiders) ** No loops, no waiting. Free tier available. [Pricing →](https://app.edgar.tools/pricing?utm_source=edgartools-docs&utm_medium=see-live&utm_content=company-insiders) Back to top --- # Entity API - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/entity-api-guide/#entitycompany-api-guide) Entity/Company API Guide ======================== This guide covers the Entity and Company API improvements in EdgarTools v5.3.0, including filer category identification and company icon retrieval. Table of Contents ----------------- 1. [Filer Category API](https://edgartools.readthedocs.io/en/latest/guides/entity-api-guide/#filer-category-api) 2. [Company Icon API](https://edgartools.readthedocs.io/en/latest/guides/entity-api-guide/#company-icon-api) 3. [Integration Examples](https://edgartools.readthedocs.io/en/latest/guides/entity-api-guide/#integration-examples) * * * Filer Category API ------------------ The SEC classifies public companies into filer categories based on their public float (market value of voting and non-voting common equity held by non-affiliates). EdgarTools v5.3.0 provides structured access to this classification data. ### Filer Status Classifications | Status | Public Float Threshold | Filing Deadlines | | --- | --- | --- | | Large Accelerated Filer | \>= $700 million | 60 days (10-K), 40 days (10-Q) | | Accelerated Filer | \>= $75M and < $700M | 75 days (10-K), 40 days (10-Q) | | Non-Accelerated Filer | < $75 million | 90 days (10-K), 45 days (10-Q) | ### Filer Qualifications In addition to the base status, companies may have these qualifications: * **Smaller Reporting Company (SRC)**: < $250M public float OR < $100M annual revenue * **Emerging Growth Company (EGC)**: < $1.235B revenue, IPO within 5 years ### Quick Usage `from edgar import Company company = Company("AAPL") # Boolean property checks if company.is_large_accelerated_filer: print("Large accelerated filer - earliest filing deadlines") if company.is_smaller_reporting_company: print("Qualifies for scaled disclosure requirements") if company.is_emerging_growth_company: print("May use EGC accommodations")` ### Available Properties | Property | Type | Description | | --- | --- | --- | | `filer_category` | `FilerCategory` | Full parsed category object | | `is_large_accelerated_filer` | `bool` | Public float >= $700M | | `is_accelerated_filer` | `bool` | Public float >= $75M and < $700M | | `is_non_accelerated_filer` | `bool` | Public float < $75M | | `is_smaller_reporting_company` | `bool` | Qualifies as SRC | | `is_emerging_growth_company` | `bool` | Qualifies as EGC | ### Working with FilerCategory Object For more detailed analysis, use the `filer_category` property: `from edgar import Company from edgar.enums import FilerStatus, FilerQualification company = Company("AAPL") category = company.filer_category # Access the base status enum status = category.status # FilerStatus.LARGE_ACCELERATED # Check specific status if category.status == FilerStatus.LARGE_ACCELERATED: print("Large accelerated filer") # Get all qualifications as a list qualifications = category.qualifications # Returns: [FilerQualification.SMALLER_REPORTING_COMPANY, ...] # String representation (original SEC format) print(str(category)) # "Large accelerated filer"` ### Parsing SEC Category Strings The SEC returns category as a compound string with `|` separator: `from edgar.enums import FilerCategory # Parse SEC format strings directly category = FilerCategory.from_string("Accelerated filer | Smaller reporting company") print(category.status) # FilerStatus.ACCELERATED print(category.is_smaller_reporting_company) # True print(category.is_emerging_growth_company) # False # Handle compound qualifications category = FilerCategory.from_string( "Non-accelerated filer | Smaller reporting company | Emerging growth company" ) print(len(category.qualifications)) # 2` ### Enums Reference `from edgar.enums import FilerStatus, FilerQualification # FilerStatus values FilerStatus.LARGE_ACCELERATED # "Large accelerated filer" FilerStatus.ACCELERATED # "Accelerated filer" FilerStatus.NON_ACCELERATED # "Non-accelerated filer" # FilerQualification values FilerQualification.SMALLER_REPORTING_COMPANY # "Smaller reporting company" FilerQualification.EMERGING_GROWTH_COMPANY # "Emerging growth company"` See it live on edgar.tools The code above checks filer categories programmatically. **edgar.tools** shows the same company metadata visually — filer status, industry classification, shares outstanding, and public float for any SEC entity. * **[See Apple's company profile →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=entity-api) ** * **[Search 940K+ SEC entities →](https://app.edgar.tools/companies?utm_source=edgartools-docs&utm_medium=see-live&utm_content=entity-api) ** Free tier available. Also includes a REST API for programmatic entity lookups. [API docs →](https://app.edgar.tools/docs?utm_source=edgartools-docs&utm_medium=see-live&utm_content=entity-api) * * * Company Icon API ---------------- EdgarTools provides access to company logo/icon images via the `get_icon_from_ticker` function. Icons are sourced from the [nvstly/icons](https://github.com/nvstly/icons) repository on GitHub. ### Basic Usage `from edgar import get_icon_from_ticker # Get icon as PNG bytes icon_bytes = get_icon_from_ticker("AAPL") if icon_bytes: # Save to file with open("apple_logo.png", "wb") as f: f.write(icon_bytes)` ### Function Signature `def get_icon_from_ticker(ticker: str) -> Optional[bytes]: """ Download an icon for a given ticker as a PNG image, if available. Args: ticker: Stock ticker symbol (e.g., "AAPL", "MSFT", "BRK-B") Returns: bytes: PNG image data if icon exists None: If no icon is available for this ticker Raises: ValueError: If ticker is invalid (empty, contains invalid characters) """` ### Handling Hyphenated Tickers As of v5.3.0, hyphenated tickers are fully supported: `# Berkshire Hathaway Class B shares icon = get_icon_from_ticker("BRK-B") # Works correctly # The function strips hyphens internally since the icon repository # stores icons as BRKB.png, not BRK-B.png` ### Validation Rules The ticker must: - Be a non-empty string - Contain only alphabetic characters (A-Z) and hyphens (-) - Not contain numbers, spaces, or special characters `# Valid tickers get_icon_from_ticker("AAPL") # OK get_icon_from_ticker("BRK-B") # OK (hyphenated) get_icon_from_ticker("msft") # OK (case insensitive) # Invalid tickers - raise ValueError get_icon_from_ticker("") # Empty string get_icon_from_ticker("AAPL123") # Contains numbers get_icon_from_ticker("AA PL") # Contains space get_icon_from_ticker(None) # Not a string` ### Caching The function uses LRU caching (maxsize=4) to avoid repeated network requests: `# First call fetches from network icon1 = get_icon_from_ticker("AAPL") # Subsequent calls return cached result icon2 = get_icon_from_ticker("AAPL") # Instant, no network call` ### Building Icon URLs If you need the URL directly (e.g., for client-side rendering): `from edgar.reference.tickers import get_ticker_icon_url url = get_ticker_icon_url("AAPL") # Returns: "https://raw.githubusercontent.com/nvstly/icons/main/ticker_icons/AAPL.png"` **Note**: For hyphenated tickers, you need to strip the hyphen manually for the URL: `ticker = "BRK-B" url = f"https://raw.githubusercontent.com/nvstly/icons/main/ticker_icons/{ticker.replace('-', '').upper()}.png" # Returns: "https://raw.githubusercontent.com/nvstly/icons/main/ticker_icons/BRKB.png"` * * * Integration Examples -------------------- ### SaaS Dashboard: Company Card Component `from edgar import Company, get_icon_from_ticker import base64 def get_company_card_data(ticker: str) -> dict: """ Build company card data for a SaaS dashboard. """ company = Company(ticker) # Get icon as base64 for embedding in HTML/JSON icon_bytes = get_icon_from_ticker(ticker) icon_base64 = base64.b64encode(icon_bytes).decode() if icon_bytes else None # Determine regulatory tier for UI badges if company.is_large_accelerated_filer: regulatory_tier = "Large Cap" tier_color = "blue" elif company.is_accelerated_filer: regulatory_tier = "Mid Cap" tier_color = "green" else: regulatory_tier = "Small Cap" tier_color = "gray" # Build badges list badges = [regulatory_tier] if company.is_smaller_reporting_company: badges.append("SRC") if company.is_emerging_growth_company: badges.append("EGC") return { "ticker": ticker, "name": company.name, "cik": company.cik, "icon_base64": icon_base64, "icon_url": f"data:image/png;base64,{icon_base64}" if icon_base64 else None, "regulatory_tier": regulatory_tier, "tier_color": tier_color, "badges": badges, "filer_category_raw": str(company.filer_category), } # Usage card = get_company_card_data("AAPL") # { # "ticker": "AAPL", # "name": "Apple Inc.", # "cik": 320193, # "icon_base64": "iVBORw0KGgo...", # "regulatory_tier": "Large Cap", # "tier_color": "blue", # "badges": ["Large Cap"], # "filer_category_raw": "Large accelerated filer" # }` ### Filtering Companies by Filer Status `from edgar import Company def filter_by_filer_status(tickers: list[str], status: str) -> list[str]: """ Filter tickers by their SEC filer status. Args: tickers: List of ticker symbols status: One of "large_accelerated", "accelerated", "non_accelerated" Returns: List of tickers matching the specified status """ results = [] for ticker in tickers: try: company = Company(ticker) match status: case "large_accelerated": if company.is_large_accelerated_filer: results.append(ticker) case "accelerated": if company.is_accelerated_filer: results.append(ticker) case "non_accelerated": if company.is_non_accelerated_filer: results.append(ticker) except Exception: continue # Skip invalid tickers return results # Find all emerging growth companies def find_egc_companies(tickers: list[str]) -> list[str]: return [t for t in tickers if Company(t).is_emerging_growth_company]` ### API Response Builder `from edgar import Company, get_icon_from_ticker from edgar.enums import FilerStatus import json def build_company_api_response(ticker: str) -> dict: """ Build a complete API response for company data. """ company = Company(ticker) category = company.filer_category return { "company": { "ticker": ticker, "name": company.name, "cik": company.cik, }, "filer_classification": { "status": category.status.value if category.status else None, "status_code": category.status.name if category.status else None, "is_large_accelerated": company.is_large_accelerated_filer, "is_accelerated": company.is_accelerated_filer, "is_non_accelerated": company.is_non_accelerated_filer, }, "qualifications": { "smaller_reporting_company": company.is_smaller_reporting_company, "emerging_growth_company": company.is_emerging_growth_company, }, "branding": { "icon_available": get_icon_from_ticker(ticker) is not None, "icon_url": f"/api/company/{ticker}/icon", # Your API endpoint }, "raw_sec_category": str(category), } # Example output for AAPL: # { # "company": {"ticker": "AAPL", "name": "Apple Inc.", "cik": 320193}, # "filer_classification": { # "status": "Large accelerated filer", # "status_code": "LARGE_ACCELERATED", # "is_large_accelerated": True, # "is_accelerated": False, # "is_non_accelerated": False # }, # "qualifications": { # "smaller_reporting_company": False, # "emerging_growth_company": False # }, # "branding": { # "icon_available": True, # "icon_url": "/api/company/AAPL/icon" # }, # "raw_sec_category": "Large accelerated filer" # }` ### Flask/FastAPI Icon Endpoint `# Flask example from flask import Flask, Response, abort from edgar import get_icon_from_ticker app = Flask(__name__) @app.route("/api/company//icon") def company_icon(ticker: str): try: icon_bytes = get_icon_from_ticker(ticker.upper()) if icon_bytes is None: abort(404, description="Icon not available for this ticker") return Response(icon_bytes, mimetype="image/png") except ValueError as e: abort(400, description=str(e))` `# FastAPI example from fastapi import FastAPI, HTTPException from fastapi.responses import Response from edgar import get_icon_from_ticker app = FastAPI() @app.get("/api/company/{ticker}/icon") async def company_icon(ticker: str): try: icon_bytes = get_icon_from_ticker(ticker.upper()) if icon_bytes is None: raise HTTPException(status_code=404, detail="Icon not available") return Response(content=icon_bytes, media_type="image/png") except ValueError as e: raise HTTPException(status_code=400, detail=str(e))` * * * Notes and Limitations --------------------- ### Filer Category API * Filer category data comes from SEC submission metadata * The `filer_category` property is cached per Company instance * Some older or unusual entities may not have category data (returns empty `FilerCategory`) ### Icon API * Icons are sourced from a third-party GitHub repository (nvstly/icons) * Not all tickers have icons available - check for `None` return * The repository focuses on popular US stocks * Icon format is PNG * Results are cached (LRU cache, maxsize=4) * Network errors (other than 404) are propagated as exceptions ### Performance Considerations `# For batch operations, consider caching at the application level from functools import lru_cache @lru_cache(maxsize=1000) def get_company_data_cached(ticker: str): company = Company(ticker) return { "name": company.name, "is_large_accelerated": company.is_large_accelerated_filer, # ... etc }` Back to top --- # Examples - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/examples/#examples) Examples ======== Learn by doing. These examples show how to solve real problems with EdgarTools. Interactive Notebooks --------------------- Run these in your browser with Google Colab -- no setup required. ### Getting Started | Notebook | Description | | --- | --- | | [First Steps](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/00_first_steps.ipynb) | Look up a company, get financials, export data | | [Getting Started](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/01_getting_started.ipynb) | Company lookup, filings, date filtering | | [Troubleshooting SSL](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/02_troubleshooting_ssl.ipynb) | Fix SSL/connection issues | ### Financial Statements | Notebook | Description | | --- | --- | | [Financial Statements](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/financial-statements-sec-python.ipynb) | Income, balance sheet, cash flow from SEC filings | | [Viewing Financial Statements](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Viewing-Financial-Statements.ipynb) | get\_financials() deep dive | | [Extract Revenue & Earnings](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/extract-revenue-earnings-python.ipynb) | Pull specific financial metrics | | [Compare Company Financials](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/compare-company-financials-python.ipynb) | Side-by-side multi-company analysis | | [Statements to DataFrame](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/financial-statements-to-dataframe.ipynb) | Export to pandas for analysis | | [XBRL Financial Data](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/xbrl-financial-data-python.ipynb) | Low-level XBRL data access | ### Company Research | Notebook | Description | | --- | --- | | [SEC EDGAR API Overview](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-edgar-api-python.ipynb) | Comprehensive library overview | | [Company Data](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-company-data-python.ipynb) | Company metadata, CIK lookup, filing history | | [Ticker Search](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Ticker-Search-with-edgartools.ipynb) | Find companies by ticker, name, or keyword | | [Industry & SIC Codes](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-industry-sic-code-python.ipynb) | Filter companies by industry | ### Filings | Notebook | Description | | --- | --- | | [Search & Filter Filings](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/search-sec-filings-python.ipynb) | Find filings by date, form type, company | | [Today's Filings](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-filings-today-python.ipynb) | Monitor current SEC filings | | [Download 10-K Reports](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/download-10k-annual-report-python.ipynb) | Parse and extract 10-K sections | | [Analyze 10-K Reports](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/analyze-10k-annual-report-python.ipynb) | Business description, risk factors, MD&A | | [10-Q Quarterly Earnings](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/10q-quarterly-earnings-python.ipynb) | Quarterly report analysis | | [8-K Earnings Releases](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/8k-earnings-release-python.ipynb) | Current event reports | | [Extract Earnings Releases](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Extract-Earnings-Releases.ipynb) | Press releases and financial tables from 8-Ks | | [Filing Text & NLP](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-filing-text-nlp-python.ipynb) | Text extraction for NLP analysis | | [Filing Exhibits](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-filing-exhibits-python.ipynb) | Work with filing attachments and exhibits | | [Bulk Download](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/download-sec-filings-bulk-python.ipynb) | Download filings in bulk | | [10-K Business Description](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/10k-business-description-python.ipynb) | Extract business overview text | | [Monitor Filings](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/monitor-sec-filings-python.ipynb) | Watch for new filings | ### Insider Trading & Ownership | Notebook | Description | | --- | --- | | [Insider Trading (Form 4)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/insider-trading-sec-form4-python.ipynb) | Track insider buys and sells | | [13F Institutional Holdings](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/13f-institutional-holdings-python.ipynb) | Fund portfolio analysis | | [Beneficial Ownership](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/beneficial-ownership-sec-python.ipynb) | 13D/G activist positions | | [Executive Compensation](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/executive-compensation-sec-python.ipynb) | Proxy statement compensation data | | [Proxy Statements (DEF 14A)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/proxy-statement-def14a-python.ipynb) | Board members, shareholder proposals | ### Funds | Notebook | Description | | --- | --- | | [ETF & Fund Holdings](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/etf-fund-holdings-python.ipynb) | ETF portfolio data | | [Mutual Fund Holdings (N-PORT)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/mutual-fund-holdings-nport-python.ipynb) | Mutual fund portfolio reports | | [Money Market Funds (N-MFP)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/money-market-fund-nmfp-python.ipynb) | Money market fund data | | [Fund Census (N-CEN)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/fund-census-ncen-python.ipynb) | Fund census reports | | [BDCs](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/bdc-business-development-company-python.ipynb) | Business Development Companies | * * * Code Examples ------------- ### Get a company's revenue in 3 lines `from edgar import Company financials = Company("AAPL").get_financials() print(f"Revenue: ${financials.get_revenue():,.0f}")` ### Compare two companies `from edgar import Company for ticker in ["AAPL", "MSFT"]: f = Company(ticker).get_financials() print(f"{ticker}: Revenue ${f.get_revenue():,.0f}, Net Income ${f.get_net_income():,.0f}")` ### Get the latest 10-K business description `from edgar import Company tenk = Company("NVDA").latest("10-K") print(tenk['Item 1'].text[:2000])` ### Get auditor and subsidiaries from a 10-K `from edgar import Company tenk = Company("AAPL").get_filings(form="10-K").latest().obj() print(tenk.auditor) # Ernst & Young LLP, PCAOB ID 42 print(tenk.subsidiaries) # SubsidiaryList from Exhibit 21` ### Track insider buying `from edgar import Company for filing in Company("AAPL").get_filings(form=4).head(10): summary = filing.obj().get_ownership_summary() if summary.primary_activity == "Purchase": print(f"{summary.insider_name}: bought {summary.net_change:,} shares")` ### Export financials to CSV `from edgar import Company financials = Company("AAPL").get_financials() df = financials.income_statement().to_dataframe() df.to_csv("apple_income.csv")` Back to top --- # Company Subsets - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/company-subsets/#company-subsets) Company Subsets =============== The `edgar.reference.company_subsets` module provides powerful and flexible tools for creating subsets of companies from SEC reference data. This is especially useful for research, analysis, educational purposes, and machine learning tasks where you need specific groups of companies. Key Features ------------ * **Exchange-based selection**: Filter by NYSE, NASDAQ, OTC, CBOE * **Popularity-based selection**: Get popular stocks, mega-cap companies, etc. * **Sampling capabilities**: Random sampling, stratified sampling, top N selection * **Filtering and combination utilities**: Include/exclude specific companies, combine sets * **Fluent interface**: Chain operations for readable, flexible subset creation * **Consistent output**: All functions return standardized DataFrames with `['cik', 'ticker', 'name', 'exchange']` columns Quick Start ----------- `from edgar.reference.company_subsets import ( CompanySubset, get_companies_by_exchanges, get_popular_companies, get_random_sample ) # Simple exchange-based selection nyse_companies = get_companies_by_exchanges('NYSE') print(f"Found {len(nyse_companies)} NYSE companies") # Get popular companies popular = get_popular_companies() print(f"Found {len(popular)} popular companies") # Random sampling random_100 = get_random_sample(n=100, random_state=42) print(f"Sampled {len(random_100)} random companies")` Fluent Interface with CompanySubset ----------------------------------- The `CompanySubset` class provides a powerful fluent interface for building complex company selections: `from edgar.reference.company_subsets import CompanySubset, PopularityTier # Complex selection with method chaining companies = (CompanySubset() .from_exchange(['NYSE', 'Nasdaq']) # Major exchanges only .exclude_tickers(['JPM', 'GS', 'C']) # Exclude some financials .sample(50, random_state=42) # Take random sample .get()) # Get the DataFrame print(f"Selected {len(companies)} companies") print(companies.head()) # Popular tech companies tech_subset = (CompanySubset() .from_popular(PopularityTier.POPULAR) # Popular companies .filter_by(lambda df: df['name'].str.contains('tech|software|computer', case=False)) .top(20, by='ticker') # Top 20 alphabetically .get())` See it live on edgar.tools The code above builds company subsets programmatically. **edgar.tools** lets you browse and search 940K+ SEC entities visually — filter by exchange, industry, or name. * **[Search the full SEC entity database →](https://app.edgar.tools/companies?utm_source=edgartools-docs&utm_medium=see-live&utm_content=company-subsets) ** * **[See Apple's company profile →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=company-subsets) ** Free tier available. Also includes a REST API for programmatic company lookups. [API docs →](https://app.edgar.tools/docs?utm_source=edgartools-docs&utm_medium=see-live&utm_content=company-subsets) Core Functions -------------- ### Exchange-Based Selection Filter companies by stock exchange: `from edgar.reference.company_subsets import get_companies_by_exchanges # Single exchange nyse_companies = get_companies_by_exchanges('NYSE') nasdaq_companies = get_companies_by_exchanges('Nasdaq') # Multiple exchanges major_exchanges = get_companies_by_exchanges(['NYSE', 'Nasdaq']) all_exchanges = get_companies_by_exchanges(['NYSE', 'Nasdaq', 'OTC', 'CBOE']) print(f"NYSE: {len(nyse_companies)} companies") print(f"NASDAQ: {len(nasdaq_companies)} companies") print(f"Major exchanges: {len(major_exchanges)} companies")` ### Popular Companies Access curated lists of popular and well-known companies: `from edgar.reference.company_subsets import get_popular_companies, PopularityTier # All popular companies all_popular = get_popular_companies() # By popularity tier mega_cap = get_popular_companies(PopularityTier.MEGA_CAP) # Top 10 popular = get_popular_companies(PopularityTier.POPULAR) # Top 50 mainstream = get_popular_companies(PopularityTier.MAINSTREAM) # Top 100 emerging = get_popular_companies(PopularityTier.EMERGING) # All available print(f"Mega cap: {len(mega_cap)} companies") print(f"Popular: {len(popular)} companies") print(f"All popular: {len(all_popular)} companies")` ### Sampling Methods Create representative samples from larger datasets: `from edgar.reference.company_subsets import ( get_random_sample, get_stratified_sample, get_top_companies_by_metric ) # Random sampling random_sample = get_random_sample(n=200, random_state=42) # Stratified sampling (maintains exchange proportions) stratified_sample = get_stratified_sample( n=100, stratify_by='exchange', random_state=42 ) # Top companies by name (alphabetical) top_alphabetical = get_top_companies_by_metric( n=50, metric='name', ascending=True ) # Sample from a specific subset nyse_random = get_random_sample( get_companies_by_exchanges('NYSE'), n=100, random_state=42 )` Filtering and Combining ----------------------- ### Include/Exclude Specific Companies `from edgar.reference.company_subsets import filter_companies, exclude_companies all_companies = get_all_companies() # Include specific tickers (FAANG companies) faang = filter_companies( all_companies, ticker_list=['META', 'AAPL', 'AMZN', 'NFLX', 'GOOGL'] ) # Include companies with names containing specific text tech_companies = filter_companies( all_companies, name_contains='Technology' ) # Include specific CIKs specific_companies = filter_companies( all_companies, cik_list=[320193, 1018724, 1652044] # AAPL, AMZN, GOOGL ) # Exclude financial companies (simplified example) non_financial = exclude_companies( all_companies, ticker_list=['JPM', 'GS', 'C', 'BAC', 'WFC'] ) # Exclude companies with 'Corp' in name non_corp = exclude_companies( all_companies, name_contains='Corp' )` ### Custom Filtering Apply custom filtering logic: `from edgar.reference.company_subsets import filter_companies # Custom filter function def large_company_filter(df): """Filter to companies with longer names (proxy for larger companies).""" return df[df['name'].str.len() > 20] # Apply custom filter large_companies = filter_companies( get_companies_by_exchanges('NYSE'), custom_filter=large_company_filter ) # Using lambda for simple filters short_tickers = filter_companies( get_popular_companies(), custom_filter=lambda df: df[df['ticker'].str.len() <= 4] )` ### Combining and Intersecting Sets `from edgar.reference.company_subsets import combine_company_sets, intersect_company_sets # Get different company sets nyse_companies = get_companies_by_exchanges('NYSE') popular_companies = get_popular_companies() tech_companies = filter_companies(get_all_companies(), name_contains='Tech') # Union: Combine multiple sets (removes duplicates) combined = combine_company_sets([nyse_companies, popular_companies, tech_companies]) # Intersection: Find companies present in all sets nyse_popular = intersect_company_sets([nyse_companies, popular_companies]) popular_tech = intersect_company_sets([popular_companies, tech_companies]) print(f"Combined: {len(combined)} companies") print(f"NYSE + Popular intersection: {len(nyse_popular)} companies") print(f"Popular + Tech intersection: {len(popular_tech)} companies")` Convenience Functions --------------------- Pre-defined functions for common company groupings: `from edgar.reference.company_subsets import ( get_faang_companies, get_tech_giants, get_dow_jones_sample ) # FAANG companies (Meta, Apple, Amazon, Netflix, Google) faang = get_faang_companies() # Major tech companies tech_giants = get_tech_giants() # Dow Jones Industrial Average sample dow_sample = get_dow_jones_sample() print(f"FAANG: {len(faang)} companies") print(f"Tech Giants: {len(tech_giants)} companies") print(f"Dow Sample: {len(dow_sample)} companies") # Display the companies print("\nFAANG Companies:") for _, company in faang.iterrows(): print(f" {company['ticker']}: {company['name']}")` Advanced Examples ----------------- ### Research Dataset Creation Create a balanced research dataset: `from edgar.reference.company_subsets import CompanySubset, PopularityTier # Create a research dataset with companies from different tiers research_dataset = [] # Get 20 mega-cap companies mega_cap = (CompanySubset() .from_popular(PopularityTier.MEGA_CAP) .sample(20, random_state=42) .get()) # Get 30 popular mid-tier companies mid_tier = (CompanySubset() .from_popular(PopularityTier.POPULAR) .exclude_tickers(mega_cap['ticker'].tolist()) # Don't overlap .sample(30, random_state=42) .get()) # Get 50 random companies from major exchanges random_companies = (CompanySubset() .from_exchange(['NYSE', 'Nasdaq']) .exclude_tickers(mega_cap['ticker'].tolist() + mid_tier['ticker'].tolist()) .sample(50, random_state=42) .get()) # Combine all for final research set research_companies = combine_company_sets([mega_cap, mid_tier, random_companies]) print(f"Research dataset: {len(research_companies)} companies") # Analyze composition exchange_dist = research_companies['exchange'].value_counts() print("\nExchange distribution:") print(exchange_dist)` ### Sector-Based Analysis Create industry-focused subsets: `# Create sector-based subsets (simplified approach using name patterns) sectors = { 'technology': ['tech', 'software', 'computer', 'digital'], 'financial': ['bank', 'financial', 'insurance', 'capital'], 'healthcare': ['health', 'medical', 'pharma', 'bio'], 'energy': ['energy', 'oil', 'gas', 'power'], 'retail': ['retail', 'store', 'market', 'shop'] } sector_companies = {} all_companies = get_companies_by_exchanges(['NYSE', 'Nasdaq']) for sector, keywords in sectors.items(): # Create pattern for all keywords pattern = '|'.join(keywords) sector_subset = filter_companies( all_companies, custom_filter=lambda df, p=pattern: df[df['name'].str.contains(p, case=False)] ) sector_companies[sector] = sector_subset print(f"{sector.title()}: {len(sector_subset)} companies") # Get top 10 from each sector for analysis analysis_set = [] for sector, companies in sector_companies.items(): top_10 = get_top_companies_by_metric(companies, n=10, metric='ticker') analysis_set.append(top_10) final_analysis_set = combine_company_sets(analysis_set) print(f"\nFinal analysis set: {len(final_analysis_set)} companies across sectors")` ### Machine Learning Dataset Preparation Prepare balanced datasets for ML training: `from edgar.reference.company_subsets import get_stratified_sample # Create training/test split with stratification all_popular = get_popular_companies() # Training set (70% of data, stratified by exchange) training_companies = get_stratified_sample( all_popular, n=int(len(all_popular) * 0.7), stratify_by='exchange', random_state=42 ) # Test set (remaining companies) test_companies = all_popular[~all_popular['cik'].isin(training_companies['cik'])] print(f"Training set: {len(training_companies)} companies") print(f"Test set: {len(test_companies)} companies") # Verify stratification worked print("\nTraining exchange distribution:") print(training_companies['exchange'].value_counts(normalize=True)) print("\nTest exchange distribution:") print(test_companies['exchange'].value_counts(normalize=True))` Data Structure -------------- All functions return a standardized pandas DataFrame with these columns: * **`cik`** (int): SEC Central Index Key - unique company identifier * **`ticker`** (str): Stock ticker symbol (e.g., 'AAPL', 'MSFT') * **`name`** (str): Official company name * **`exchange`** (str): Stock exchange ('NYSE', 'Nasdaq', 'OTC', 'CBOE', etc.) `# Example output structure companies = get_random_sample(5) print(companies) # cik ticker name exchange # 0 320193 AAPL Apple Inc. Nasdaq # 1 1018724 AMZN Amazon.com, Inc. Nasdaq # 2 1652044 GOOGL Alphabet Inc. Nasdaq # 3 789019 MSFT Microsoft Corporation Nasdaq # 4 1326801 META Meta Platforms, Inc Nasdaq` Error Handling -------------- The module includes robust error handling and logging: `# Functions gracefully handle errors and return empty DataFrames empty_result = get_companies_by_exchanges('INVALID_EXCHANGE') print(f"Invalid exchange result: {len(empty_result)} companies") # Check for empty results companies = get_random_sample(n=10) if companies.empty: print("No companies found") else: print(f"Found {len(companies)} companies") # All functions include logging for debugging import logging logging.basicConfig(level=logging.DEBUG) # Now function calls will show debug information companies = get_popular_companies()` Performance Considerations -------------------------- * **Caching**: `get_all_companies()` uses LRU cache for performance * **Lazy evaluation**: CompanySubset operations are efficient and don't duplicate data unnecessarily * **Memory efficient**: Functions work with DataFrame views when possible * **Batch operations**: Use combine/intersect functions instead of loops for better performance `# Efficient: Use batch operations company_sets = [ get_companies_by_exchanges('NYSE'), get_companies_by_exchanges('Nasdaq'), get_popular_companies() ] combined = combine_company_sets(company_sets) # Less efficient: Multiple individual operations in loops # combined = pd.DataFrame() # for exchange in ['NYSE', 'Nasdaq']: # exchange_companies = get_companies_by_exchanges(exchange) # combined = pd.concat([combined, exchange_companies]) # Avoid this pattern` Integration with Edgar Tools ---------------------------- Company subsets integrate seamlessly with other Edgar tools: `from edgar import Company from edgar.reference.company_subsets import get_tech_giants # Get tech companies and analyze their latest filings tech_companies = get_tech_giants() for _, company_info in tech_companies.head(5).iterrows(): try: company = Company(company_info['ticker']) latest_filing = company.get_filings(form='10-K').latest() print(f"{company_info['ticker']}: Latest 10-K filed {latest_filing.filing_date}") except: print(f"{company_info['ticker']}: No recent 10-K found")` Best Practices -------------- 1. **Use appropriate sample sizes**: Don't sample more companies than you need for analysis 2. **Set random seeds**: Use `random_state` parameter for reproducible results 3. **Handle empty results**: Always check if returned DataFrames are empty 4. **Combine operations efficiently**: Use method chaining with CompanySubset for readable code 5. **Cache results**: Store company subsets if you'll reuse them multiple times 6. **Validate data**: Check that your filters return expected results `# Good: Reproducible and efficient companies = (CompanySubset() .from_exchange('NYSE') .sample(100, random_state=42) .get()) # Store for reuse cached_companies = companies.copy() # Good: Check for empty results if not companies.empty: print(f"Analysis ready with {len(companies)} companies") else: print("No companies found matching criteria")` This module provides a comprehensive toolkit for creating company subsets tailored to your specific research, analysis, or educational needs. The combination of simple functions and the powerful fluent interface makes it easy to create both simple selections and complex, multi-criteria company datasets. Back to top --- # Examples - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/examples/#examples) Examples ======== Learn by doing. These examples show how to solve real problems with EdgarTools. Interactive Notebooks --------------------- Run these in your browser with Google Colab -- no setup required. ### Getting Started | Notebook | Description | | --- | --- | | [First Steps](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/00_first_steps.ipynb) | Look up a company, get financials, export data | | [Getting Started](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/01_getting_started.ipynb) | Company lookup, filings, date filtering | | [Troubleshooting SSL](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/02_troubleshooting_ssl.ipynb) | Fix SSL/connection issues | ### Financial Statements | Notebook | Description | | --- | --- | | [Financial Statements](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/financial-statements-sec-python.ipynb) | Income, balance sheet, cash flow from SEC filings | | [Viewing Financial Statements](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Viewing-Financial-Statements.ipynb) | get\_financials() deep dive | | [Extract Revenue & Earnings](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/extract-revenue-earnings-python.ipynb) | Pull specific financial metrics | | [Compare Company Financials](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/compare-company-financials-python.ipynb) | Side-by-side multi-company analysis | | [Statements to DataFrame](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/financial-statements-to-dataframe.ipynb) | Export to pandas for analysis | | [XBRL Financial Data](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/xbrl-financial-data-python.ipynb) | Low-level XBRL data access | ### Company Research | Notebook | Description | | --- | --- | | [SEC EDGAR API Overview](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-edgar-api-python.ipynb) | Comprehensive library overview | | [Company Data](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-company-data-python.ipynb) | Company metadata, CIK lookup, filing history | | [Ticker Search](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Ticker-Search-with-edgartools.ipynb) | Find companies by ticker, name, or keyword | | [Industry & SIC Codes](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-industry-sic-code-python.ipynb) | Filter companies by industry | ### Filings | Notebook | Description | | --- | --- | | [Search & Filter Filings](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/search-sec-filings-python.ipynb) | Find filings by date, form type, company | | [Today's Filings](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-filings-today-python.ipynb) | Monitor current SEC filings | | [Download 10-K Reports](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/download-10k-annual-report-python.ipynb) | Parse and extract 10-K sections | | [Analyze 10-K Reports](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/analyze-10k-annual-report-python.ipynb) | Business description, risk factors, MD&A | | [10-Q Quarterly Earnings](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/10q-quarterly-earnings-python.ipynb) | Quarterly report analysis | | [8-K Earnings Releases](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/8k-earnings-release-python.ipynb) | Current event reports | | [Extract Earnings Releases](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/Extract-Earnings-Releases.ipynb) | Press releases and financial tables from 8-Ks | | [Filing Text & NLP](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-filing-text-nlp-python.ipynb) | Text extraction for NLP analysis | | [Filing Exhibits](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/sec-filing-exhibits-python.ipynb) | Work with filing attachments and exhibits | | [Bulk Download](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/download-sec-filings-bulk-python.ipynb) | Download filings in bulk | | [10-K Business Description](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/10k-business-description-python.ipynb) | Extract business overview text | | [Monitor Filings](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/monitor-sec-filings-python.ipynb) | Watch for new filings | ### Insider Trading & Ownership | Notebook | Description | | --- | --- | | [Insider Trading (Form 4)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/insider-trading-sec-form4-python.ipynb) | Track insider buys and sells | | [13F Institutional Holdings](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/13f-institutional-holdings-python.ipynb) | Fund portfolio analysis | | [Beneficial Ownership](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/beneficial-ownership-sec-python.ipynb) | 13D/G activist positions | | [Executive Compensation](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/executive-compensation-sec-python.ipynb) | Proxy statement compensation data | | [Proxy Statements (DEF 14A)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/proxy-statement-def14a-python.ipynb) | Board members, shareholder proposals | ### Funds | Notebook | Description | | --- | --- | | [ETF & Fund Holdings](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/etf-fund-holdings-python.ipynb) | ETF portfolio data | | [Mutual Fund Holdings (N-PORT)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/mutual-fund-holdings-nport-python.ipynb) | Mutual fund portfolio reports | | [Money Market Funds (N-MFP)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/money-market-fund-nmfp-python.ipynb) | Money market fund data | | [Fund Census (N-CEN)](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/fund-census-ncen-python.ipynb) | Fund census reports | | [BDCs](https://colab.research.google.com/github/dgunning/edgartools/blob/main/notebooks/bdc-business-development-company-python.ipynb) | Business Development Companies | * * * Code Examples ------------- ### Get a company's revenue in 3 lines `from edgar import Company financials = Company("AAPL").get_financials() print(f"Revenue: ${financials.get_revenue():,.0f}")` ### Compare two companies `from edgar import Company for ticker in ["AAPL", "MSFT"]: f = Company(ticker).get_financials() print(f"{ticker}: Revenue ${f.get_revenue():,.0f}, Net Income ${f.get_net_income():,.0f}")` ### Get the latest 10-K business description `from edgar import Company tenk = Company("NVDA").latest("10-K") print(tenk['Item 1'].text[:2000])` ### Get auditor and subsidiaries from a 10-K `from edgar import Company tenk = Company("AAPL").get_filings(form="10-K").latest().obj() print(tenk.auditor) # Ernst & Young LLP, PCAOB ID 42 print(tenk.subsidiaries) # SubsidiaryList from Exhibit 21` ### Track insider buying `from edgar import Company for filing in Company("AAPL").get_filings(form=4).head(10): summary = filing.obj().get_ownership_summary() if summary.primary_activity == "Purchase": print(f"{summary.insider_name}: bought {summary.net_change:,} shares")` ### Export financials to CSV `from edgar import Company financials = Company("AAPL").get_financials() df = financials.income_statement().to_dataframe() df.to_csv("apple_income.csv")` Back to top --- # Attachments - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/filing-attachments/#attachments) Attachments =========== Once you have a `Filing` instance you can access the attachments for the filing using the `attachments` property. `filing.attachments` ![attachments](https://raw.githubusercontent.com/dgunning/edgartools/main/docs/images/attachments.png) ### Auto-Parsed Exhibits Some exhibit types are automatically parsed when accessed through a data object: * **EX-21** (Subsidiaries): `tenk.subsidiaries` returns a `SubsidiaryList` with name, jurisdiction, and ownership percentage for each subsidiary. ### Get an attachment by index You can get an attachment by index using the `[]` operator and using the `Seq` number of the attachment. The primary filing document is always at index **1**, and is usually HTML or XML. `attachment = filing.attachments[1] attachment` ![attachments](https://raw.githubusercontent.com/dgunning/edgartools/main/docs/images/snowflake-attachments.png) ### Viewing an attachment You can view the attachment in a browser using the `view()` method. This works if the attachment is a text or html file. `attachment.view()` ![attachments](https://raw.githubusercontent.com/dgunning/edgartools/main/docs/images/view-attachment.png) This extracts the text of the attachment and renders it in the console. If you need to get the text use the `text()` method. ### Getting the text content of an attachment You can get the text content of an attachment using the `text()` function. `text = attachment.text() print(text)` This will print the text content of the attachment. ### Converting HTML attachments to markdown You can convert HTML attachments to markdown format using the `markdown()` method. `# Convert a single HTML attachment to markdown attachment = filing.attachments[1] # Get the primary document if attachment.is_html(): markdown_content = attachment.markdown() print(markdown_content)` The `markdown()` method returns `None` for non-HTML attachments, so you can safely call it on any attachment. #### Page Break Delimiter Support The `markdown()` method supports optional page break delimiters to help you understand document structure: `# Convert with page break delimiters attachment = filing.attachments[1] markdown_with_breaks = attachment.markdown(include_page_breaks=True) # Page breaks appear as: {1}------------------------------------------------ # Where the number indicates the page number` When `include_page_breaks=True`, the markdown will include delimiters at page boundaries in the format: - `{0}------------------------------------------------` at the start of the document - `{1}------------------------------------------------` before the second page content - `{2}------------------------------------------------` before the third page content - And so on... ##### Customizing Page Numbering You can control the starting page number for page break markers using the `start_page_number` parameter: `# Start page numbering at 1 (instead of 0) attachment = filing.attachments[1] markdown_with_breaks = attachment.markdown(include_page_breaks=True, start_page_number=1) # This will produce: {1}------------------------------------------------, {2}------------------------------------------------, etc. # Start page numbering at 5 markdown_with_breaks = attachment.markdown(include_page_breaks=True, start_page_number=5) # This will produce: {5}------------------------------------------------, {6}------------------------------------------------, etc.` This is particularly useful when you want to align page numbers with external document numbering or when processing documents that are part of a larger collection. ### Batch markdown conversion You can convert all HTML attachments in a filing to markdown at once: `# Convert all HTML attachments (without page breaks) markdown_dict = filing.attachments.markdown() # Convert all HTML attachments with page breaks markdown_dict = filing.attachments.markdown(include_page_breaks=True) # Convert all HTML attachments with page breaks starting at page 1 markdown_dict = filing.attachments.markdown(include_page_breaks=True, start_page_number=1) # Result is a dictionary: {"filename.htm": "markdown content", ...} for filename, content in markdown_dict.items(): print(f"--- {filename} ---") print(content[:500]) # Show first 500 characters` ### Saving markdown content You can save the markdown content to files: `# Save individual attachment markdown attachment = filing.attachments[1] markdown_content = attachment.markdown() if markdown_content: with open(f"{attachment.document}.md", "w") as f: f.write(markdown_content) # Save all HTML attachments as markdown files markdown_dict = filing.attachments.markdown() for doc_name, markdown_content in markdown_dict.items(): # Remove extension and add .md base_name = doc_name.rsplit('.', 1)[0] with open(f"{base_name}.md", "w") as f: f.write(markdown_content)` ### Downloading an attachment You can download the attachment using the `download()` method. This will download the attachment to the current working directory. `attachment.download('/path/to/download')` If the path is a directory the attachment will be downloaded to that directory using the original name of the file. If the path is a file the attachment will be downloaded to that file. This allows you to rename the attachment. If you don't provide a path the content of the attachment will be returned as a string. Back to top --- # Attachments - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/filing-attachments/#attachments) Attachments =========== Once you have a `Filing` instance you can access the attachments for the filing using the `attachments` property. `filing.attachments` ![attachments](https://raw.githubusercontent.com/dgunning/edgartools/main/docs/images/attachments.png) ### Auto-Parsed Exhibits Some exhibit types are automatically parsed when accessed through a data object: * **EX-21** (Subsidiaries): `tenk.subsidiaries` returns a `SubsidiaryList` with name, jurisdiction, and ownership percentage for each subsidiary. ### Get an attachment by index You can get an attachment by index using the `[]` operator and using the `Seq` number of the attachment. The primary filing document is always at index **1**, and is usually HTML or XML. `attachment = filing.attachments[1] attachment` ![attachments](https://raw.githubusercontent.com/dgunning/edgartools/main/docs/images/snowflake-attachments.png) ### Viewing an attachment You can view the attachment in a browser using the `view()` method. This works if the attachment is a text or html file. `attachment.view()` ![attachments](https://raw.githubusercontent.com/dgunning/edgartools/main/docs/images/view-attachment.png) This extracts the text of the attachment and renders it in the console. If you need to get the text use the `text()` method. ### Getting the text content of an attachment You can get the text content of an attachment using the `text()` function. `text = attachment.text() print(text)` This will print the text content of the attachment. ### Converting HTML attachments to markdown You can convert HTML attachments to markdown format using the `markdown()` method. `# Convert a single HTML attachment to markdown attachment = filing.attachments[1] # Get the primary document if attachment.is_html(): markdown_content = attachment.markdown() print(markdown_content)` The `markdown()` method returns `None` for non-HTML attachments, so you can safely call it on any attachment. #### Page Break Delimiter Support The `markdown()` method supports optional page break delimiters to help you understand document structure: `# Convert with page break delimiters attachment = filing.attachments[1] markdown_with_breaks = attachment.markdown(include_page_breaks=True) # Page breaks appear as: {1}------------------------------------------------ # Where the number indicates the page number` When `include_page_breaks=True`, the markdown will include delimiters at page boundaries in the format: - `{0}------------------------------------------------` at the start of the document - `{1}------------------------------------------------` before the second page content - `{2}------------------------------------------------` before the third page content - And so on... ##### Customizing Page Numbering You can control the starting page number for page break markers using the `start_page_number` parameter: `# Start page numbering at 1 (instead of 0) attachment = filing.attachments[1] markdown_with_breaks = attachment.markdown(include_page_breaks=True, start_page_number=1) # This will produce: {1}------------------------------------------------, {2}------------------------------------------------, etc. # Start page numbering at 5 markdown_with_breaks = attachment.markdown(include_page_breaks=True, start_page_number=5) # This will produce: {5}------------------------------------------------, {6}------------------------------------------------, etc.` This is particularly useful when you want to align page numbers with external document numbering or when processing documents that are part of a larger collection. ### Batch markdown conversion You can convert all HTML attachments in a filing to markdown at once: `# Convert all HTML attachments (without page breaks) markdown_dict = filing.attachments.markdown() # Convert all HTML attachments with page breaks markdown_dict = filing.attachments.markdown(include_page_breaks=True) # Convert all HTML attachments with page breaks starting at page 1 markdown_dict = filing.attachments.markdown(include_page_breaks=True, start_page_number=1) # Result is a dictionary: {"filename.htm": "markdown content", ...} for filename, content in markdown_dict.items(): print(f"--- {filename} ---") print(content[:500]) # Show first 500 characters` ### Saving markdown content You can save the markdown content to files: `# Save individual attachment markdown attachment = filing.attachments[1] markdown_content = attachment.markdown() if markdown_content: with open(f"{attachment.document}.md", "w") as f: f.write(markdown_content) # Save all HTML attachments as markdown files markdown_dict = filing.attachments.markdown() for doc_name, markdown_content in markdown_dict.items(): # Remove extension and add .md base_name = doc_name.rsplit('.', 1)[0] with open(f"{base_name}.md", "w") as f: f.write(markdown_content)` ### Downloading an attachment You can download the attachment using the `download()` method. This will download the attachment to the current working directory. `attachment.download('/path/to/download')` If the path is a directory the attachment will be downloaded to that directory using the original name of the file. If the path is a file the attachment will be downloaded to that file. This allows you to rename the attachment. If you don't provide a path the content of the attachment will be returned as a string. Back to top --- # Cheat Sheet - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/quick-guide/#cheat-sheet) Cheat Sheet =========== Common EdgarTools operations at a glance. For a step-by-step introduction, see the [Quick Start](https://edgartools.readthedocs.io/en/stable/quickstart/) . ### Setup | | Code | | --- | --- | | Set your EDGAR identity in Linux/Mac | `export EDGAR_IDENTITY="email@domain.com"` | | Set your EDGAR identity in Windows | `set EDGAR_IDENTITY="email@domain.com"` | | Set identity in Windows Powershell | `$env:EDGAR_IDENTITY="email@domain.com"` | | Set identity in Python | `set_identity("email@domain.com")` | | Importing the library | `from edgar import *` | ### Working with a company 🏢 > See also: [Find a Company](https://edgartools.readthedocs.io/en/stable/guides/finding-companies/) | | Code | | --- | --- | | 🔍 Get a company by ticker | `company = Company("AAPL")` | | 🔍 Get a company by CIK | `company = Company("0000320193")` | | 🔎 Find filings by form and ticker | `find(form="10-K", ticker="AAPL")` | | 📊 Get shares outstanding | `company.shares_outstanding` | | 💰 Get public float | `company.public_float` | | 🏭 Get industry | `company.industry` | | 📋 Get company facts | `company.get_facts()` | | 🐼 Get company facts as a DataFrame | `company.get_facts().to_pandas()` | ### Financial statements 💵 > See also: [Financial Statements Guide](https://edgartools.readthedocs.io/en/stable/guides/financial-data/) | | Code | | --- | --- | | 📊 Get a company's financials | `financials = company.get_financials()` | | 📈 Get the income statement | `financials.income_statement()` | | 🏦 Get the balance sheet | `financials.balance_sheet()` | | 💸 Get the cash flow statement | `financials.cashflow_statement()` | | 💰 Get revenue | `financials.get_revenue()` | | 💵 Get net income | `financials.get_net_income()` | | 📊 Get operating income | `financials.get_operating_income()` | | 🐼 Export statement to DataFrame | `financials.income_statement().to_dataframe()` | ### Working with filings 📁 > See also: [Working with Filings](https://edgartools.readthedocs.io/en/stable/guides/working-with-filing/) > · [Search & Filter](https://edgartools.readthedocs.io/en/stable/guides/searching-filings/) #### 🔍 Getting Filings | | Code | | --- | --- | | 📅 Get filings for the year to date | `filings = get_filings()` | | 📊 Get only XBRL filings | `filings = get_filings(index="xbrl")` | | 📆 Get filings for a specific year | `filings = get_filings(2020)` | | 🗓️ Get filings for a specific quarter | `filings = get_filings(2020, 1)` | | 📚 Get filings for multiple years | `filings = get_filings([2020, 2021])` | | 📈 Get filings for a range of years | `filings = get_filings(year=range(2010, 2020))` | | 📈 Get filings released just now | `filings = get_latest_filings()` | #### 📄 Filtering Filings | | Code | | --- | --- | | 📝 Filter by form type | `filings.filter(form="10-K")` | | 📑 Filter by multiple forms | `filings.filter(form=["10-K", "10-Q"])` | | 🔄 Include form amendments | `filings.filter(form="10-K", amendments=True)` | | 🏢 Filter by CIK | `filings.filter(cik="0000320193")` | | 🏙️ Filter by multiple CIKs | `filings.filter(cik=["0000320193", "1018724"])` | | 🏷️ Filter by ticker | `filings.filter(ticker="AAPL")` | | 🏷️🏷️ Filter by multiple tickers | `filings.filter(ticker=["AAPL", "MSFT"])` | | 📅 Filter on a specific date | `filings.filter(date="2020-01-01")` | | 📅↔️📅 Filter between dates | `filings.filter(date="2020-01-01:2020-03-01")` | | 📅⬅️ Filter before a date | `filings.filter(date=":2020-03-01")` | | 📅➡️ Filter after a date | `filings.filter(date="2020-03-01:")` | | 🔀 Combine multiple filters | `filings.filter(form="10-K", date="2020-01-01:", ticker="AAPL")` | #### 📊 Viewing and Manipulating Filings | | Code | | --- | --- | | ⏭️ Show the next page of filings | `filings.next()` | | ⏮️ Show the previous page of filings | `filings.previous()` | | 🔝 Get the first n filings | `filings.head(20)` | | 🔚 Get the last n filings | `filings.tail(20)` | | 🕒 Get the latest n filings by date | `filings.latest(20)` | | 🎲 Get a random sample of filings | `filings.sample(20)` | | 🐼 Get filings as a pandas DataFrame | `filings.to_pandas()` | ### Company filings 📂 > See also: [Find a Company](https://edgartools.readthedocs.io/en/stable/guides/finding-companies/) | | Code | | --- | --- | | 📁 Get company filings | `company.get_filings()` | | 📝 Get company filings by form | `company.get_filings(form="10-K")` | | 🕒 Get the latest 10-Q | `company.latest("10-Q")` | | 📑 Get the last 5 10-Qs | `company.get_filings(form="10-Q").head(5)` | | 🔢 Get a filing by accession number | `company.get_filing(accession_number="0000320193-21-000139")` | ### Working with a filing 📄 > See also: [Working with Filings](https://edgartools.readthedocs.io/en/stable/guides/working-with-filing/) #### 🔍 Accessing and Viewing a Filing | | Code | | --- | --- | | 📌 Get a single filing | `filing = filings[3]` | | 🔢 Get a filing by accession number | `filing = get_by_accession_number("0000320193-20-34576")` | | 🏠 Get the filing homepage | `filing.homepage` | | 🌐 Open a filing in the browser | `filing.open()` | | 🏠 Open homepage in the browser | `filing.homepage.open()` | | 💻 View the filing in the terminal | `filing.view()` | #### 📊 Extracting Filing Content | | Code | | --- | --- | | 🌐 Get the HTML of the filing | `filing.html()` | | 📊 Get the XBRL of the filing | `filing.xbrl()` | | 📝 Get the filing as markdown | `filing.markdown()` | | 📄 Get the full submission text | `filing.full_text_submission()` | | 🔍 Preview data object type | `filing.obj_type` | | 🔢 Get and parse filing data object | `filing.obj()` | | 📑 Get filing header | `filing.header` | #### 🔎 Searching Inside a Filing | | Code | | --- | --- | | 🔍 Search within the filing | `filing.search("query")` | | 🔍 Search with regex | `filing.search("pattern", regex=True)` | | 📊 Get filing sections | `filing.sections()` | #### 📎 Working with Attachments > See also: [Filing Attachments](https://edgartools.readthedocs.io/en/stable/guides/filing-attachments/) | | Code | | --- | --- | | 📁 Get all filing attachments | `filing.attachments` | | 📄 Get a single attachment | `attachment = filing.attachments[0]` | | 🌐 Open attachment in browser | `attachment.open()` | | ⬇️ Download an attachment | `content = attachment.download()` | ### 10-K Annual Report data 📊 > See also: [Working with Filings](https://edgartools.readthedocs.io/en/stable/guides/working-with-filing/) | | Code | | --- | --- | | 📄 Get 10-K as data object | `tenk = company.get_filings(form="10-K").latest().obj()` | | 🏢 Get auditor information | `tenk.auditor` | | 🏢 Get auditor name | `tenk.auditor.name` | | 🔢 Get PCAOB firm ID | `tenk.auditor.firm_id` | | 🏗️ Get subsidiaries | `tenk.subsidiaries` | | 🐼 Subsidiaries as DataFrame | `tenk.subsidiaries.to_dataframe()` | ### Proxy statements (executive compensation) 💼 > See also: [Proxy Statements Guide](https://edgartools.readthedocs.io/en/stable/guides/proxystatement-data-object-guide/) | | Code | | --- | --- | | 📋 Get latest proxy statement | `proxy = company.get_filings(form="DEF 14A").latest().obj()` | | 👤 Get CEO name | `proxy.peo_name` | | 💰 Get CEO total compensation | `proxy.peo_total_comp` | | 📊 Get 5-year exec compensation DataFrame | `proxy.executive_compensation` | | 📈 Get pay vs performance DataFrame | `proxy.pay_vs_performance` | | 📉 Get company TSR | `proxy.total_shareholder_return` | | 📉 Get peer group TSR | `proxy.peer_group_tsr` | Prefer a visual interface? Every operation above also works through **[edgar.tools](https://app.edgar.tools/?utm_source=edgartools-docs&utm_medium=see-live&utm_content=cheat-sheet) ** — the same SEC data in a web UI, no code required. * **[Browse any company's filings and financials →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=cheat-sheet) ** * **[Watch filings arrive in real time →](https://app.edgar.tools/filings?utm_source=edgartools-docs&utm_medium=see-live&utm_content=cheat-sheet) ** * **[Search disclosures across 12 XBRL topics →](https://app.edgar.tools/disclosures?utm_source=edgartools-docs&utm_medium=see-live&utm_content=cheat-sheet) ** Also includes a REST API (20+ endpoints), hosted MCP server, and data exports. Free tier: 100 API calls/day. Back to top --- # Filter by Criteria - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/filtering-filings/#filter-sec-filings-by-form-type-date-ticker-exchange-and-cik) Filter SEC Filings — By Form Type, Date, Ticker, Exchange, and CIK ================================================================== Learn how to filter SEC filings by multiple criteria to find exactly what you need. Two Ways to Filter ------------------ You can filter filings in two ways: 1. **Filter while getting** - Use `get_filings()` parameters to filter from all SEC filings 2. **Filter after getting** - Use `.filter()` method to refine an existing `Filings` collection Both approaches work similarly, but filtering while getting is more efficient when you know the criteria upfront. Filter While Getting Filings ---------------------------- ### Filter by Form Type Get filings of a specific SEC form: `from edgar import get_filings # Single form type tenk = get_filings(2024, 1, form="10-K") # Multiple form types financial = get_filings(2024, 1, form=["10-K", "10-Q"]) # Proxy statements proxies = get_filings(2024, 1, form="DEF 14A")` ### Include or Exclude Amendments By default, amendments are included. Exclude them with `amendments=False`: `# Include amendments (default) all_10k = get_filings(2024, 1, form="10-K", amendments=True) # Exclude amendments original_only = get_filings(2024, 1, form="10-K", amendments=False)` ### Filter by Date #### Specific Date `# Filings on a specific date jan_15 = get_filings(2024, 1, filing_date="2024-01-15")` #### Date Range `# Filings between two dates jan_filings = get_filings(2024, 1, filing_date="2024-01-01:2024-01-31") # Q1 2024 q1 = get_filings(2024, filing_date="2024-01-01:2024-03-31")` #### Open-Ended Ranges `# From date onwards recent = get_filings(2024, 1, filing_date="2024-01-15:") # Up to a date older = get_filings(2024, 1, filing_date=":2024-01-15")` ### Combine Filters `# 10-K filings from January 2024, no amendments filings = get_filings( year=2024, quarter=1, form="10-K", filing_date="2024-01-01:2024-01-31", amendments=False )` See it live on edgar.tools The code above filters filings by form, date, and amendments. **edgar.tools** provides the same filtering in a visual interface — combine criteria and see results update in real time. * **[Filter the real-time filing stream →](https://app.edgar.tools/filings?utm_source=edgartools-docs&utm_medium=see-live&utm_content=filtering-filings) ** * **[Browse Apple's filing history →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=filtering-filings) ** Also available via REST API with form type, date range, and ticker filters. Free tier: 100 API calls/day. Filter After Getting Filings ---------------------------- Use the `.filter()` method to refine an existing collection: ### Filter by Form `filings = get_filings(2024, 1) # Filter to 10-K only tenk = filings.filter(form="10-K") # Multiple forms financial = filings.filter(form=["10-K", "10-Q"])` ### Filter by Date `filings = get_filings(2024, 1) # Specific date jan_1 = filings.filter(date="2024-01-01") # Date range jan_range = filings.filter(date="2024-01-01:2024-01-31") # From date onwards recent = filings.filter(date="2024-01-15:")` ### Filter by Company (CIK) `filings = get_filings(2024, 1) # Filter by CIK (integer) apple = filings.filter(cik=320193) # Filter by CIK (string) apple = filings.filter(cik="0000320193") # Multiple companies faang = filings.filter(cik=[320193, 1318605, 1652044])` ### Filter by Ticker `filings = get_filings(2024, 1) # Single ticker apple = filings.filter(ticker="AAPL") # Multiple tickers tech = filings.filter(ticker=["AAPL", "MSFT", "GOOGL", "AMZN"])` **Note:** Ticker filtering performs a CIK lookup first. If you know the CIK, use it directly for better performance. ### Filter by Exchange `filings = get_filings(2024, 1) # Single exchange nasdaq = filings.filter(exchange="NASDAQ") # Multiple exchanges major = filings.filter(exchange=["NASDAQ", "NYSE"])` **Available exchanges:** - NASDAQ - NYSE - CBOE - OTC ### Filter by Accession Number `filings = get_filings(2024, 1) # Single accession number filing = filings.filter(accession_number="0000320193-24-000001") # Multiple accession numbers specific = filings.filter(accession_number=[ "0000320193-24-000001", "0001318605-24-000001" ])` ### Filter Amendments `filings = get_filings(2024, 1, form="10-K") # Exclude amendments original_only = filings.filter(amendments=False) # Only amendments amendments_only = filings.filter(amendments=True)` Chain Filters ------------- Build complex queries by chaining multiple filters: `from edgar import get_filings # Start with all Q1 2024 filings filings = get_filings(2024, 1) # Chain filters for specificity result = (filings .filter(form="10-K") .filter(exchange="NASDAQ") .filter(date="2024-01-01:2024-01-31") .filter(amendments=False)) print(f"Found {len(result)} filings matching all criteria")` Alternatively, combine multiple criteria in one filter: `result = filings.filter( form="10-K", exchange="NASDAQ", date="2024-01-01:2024-01-31", amendments=False )` Use head, tail, and sample -------------------------- Limit results after filtering: ### head() Get the first n filings: `filings = get_filings(2024, 1, form="10-K") # Get first 10 first_10 = filings.head(10)` ### tail() Get the last n filings: `# Get last 10 last_10 = filings.tail(10)` ### sample() Get a random sample: `# Get random sample of 10 random_10 = filings.sample(10)` ### latest() Get most recent filings: `# Get latest single filing latest = filings.latest() # Get latest 20 filings latest_20 = filings.latest(20)` Search by Company Name ---------------------- Use `.find()` to search by company name: `filings = get_filings(2024, 1) # Find companies with "Technology" in name tech = filings.find("Technology") # Find specific company apple = filings.find("Apple") # Case-insensitive partial match results = filings.find("tesla")` Common Filtering Patterns ------------------------- ### Get Latest 10-K for NASDAQ Companies `from edgar import get_filings filings = get_filings(2024, 1, form="10-K") nasdaq = filings.filter(exchange="NASDAQ") latest_20 = nasdaq.latest(20) for filing in latest_20: print(f"{filing.company}: {filing.filing_date}")` ### Get All 8-K Filings for Specific Companies `filings = get_filings(2024, 1, form="8-K") # Filter to FAANG companies faang = filings.filter(ticker=["AAPL", "AMZN", "NFLX", "GOOGL", "META"]) print(f"Found {len(faang)} 8-K filings from FAANG")` ### Get Financial Reports from Tech Companies in January `# Get all Q1 filings filings = get_filings(2024, 1) # Filter to financial reports financial = filings.filter(form=["10-K", "10-Q"]) # Filter to NASDAQ (proxy for tech-heavy) nasdaq = financial.filter(exchange="NASDAQ") # Filter to January only jan = nasdaq.filter(date="2024-01-01:2024-01-31") print(f"Found {len(jan)} NASDAQ financial reports in January")` ### Get Original 10-K Filings (No Amendments) `filings = get_filings(2024, 1, form="10-K", amendments=False) # Or filter an existing collection all_10k = get_filings(2024, 1, form="10-K") original = all_10k.filter(amendments=False)` ### Get Filings by Year and Quarter Combinations `# Single year, single quarter q1_2024 = get_filings(2024, 1) # Single year, multiple quarters h1_2024 = get_filings(2024, [1, 2]) # Multiple years, single quarter q4_multi_year = get_filings([2022, 2023, 2024], 4) # Multiple years, all quarters multi_year = get_filings([2022, 2023, 2024]) # Year range range_2020_2024 = get_filings(range(2020, 2025)) # 2020-2024` Export Filtered Results ----------------------- ### To DataFrame `filings = get_filings(2024, 1, form="10-K") nasdaq = filings.filter(exchange="NASDAQ") # Convert to DataFrame df = nasdaq.to_pandas() # Or select specific columns df = nasdaq.to_pandas('company', 'filing_date', 'cik', 'accession_no') print(df.head())` ### To Parquet `filings = get_filings(2024, 1, form="10-K") nasdaq = filings.filter(exchange="NASDAQ") # Save as parquet nasdaq.save_parquet("nasdaq_10k_q1_2024.parquet")` Performance Tips ---------------- ### Filter Early **Efficient:** `# Filter using get_filings parameters filings = get_filings(2024, 1, form="10-K")` **Less Efficient:** `# Get everything then filter filings = get_filings(2024, 1).filter(form="10-K")` ### Use CIK Instead of Ticker **Efficient:** `# Filter by CIK (direct lookup) filings = filings.filter(cik=320193)` **Less Efficient:** `# Filter by ticker (requires CIK lookup first) filings = filings.filter(ticker="AAPL")` ### Limit Results Early `# Get only what you need filings = get_filings(2024, 1, form="10-K").head(50) # Better than processing all then limiting all_filings = get_filings(2024, 1, form="10-K") # ... process all ... limited = all_filings.head(50)` Error Handling -------------- `from edgar import get_filings try: filings = get_filings(2024, 1, form="10-K") if filings.empty: print("No filings found") else: # Filter nasdaq = filings.filter(exchange="NASDAQ") if nasdaq.empty: print("No NASDAQ filings") else: print(f"Found {len(nasdaq)} NASDAQ 10-K filings") except Exception as e: print(f"Error: {e}")` See Also -------- * **[Filings API Reference](https://edgartools.readthedocs.io/en/stable/api/filings/) ** - Complete Filings class documentation * **[Filing API Reference](https://edgartools.readthedocs.io/en/stable/api/filing/) ** - Individual filing operations * **[Search Filings Guide](https://edgartools.readthedocs.io/en/stable/guides/searching-filings/) ** - Finding specific filings * **[Current Filings Guide](https://edgartools.readthedocs.io/en/stable/guides/current-filings/) ** - Access today's filings * **[Working with Filings](https://edgartools.readthedocs.io/en/stable/guides/working-with-filing/) ** - Extract data from filings Back to top --- # Reference Data - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/reference-data/#sec-reference-data) SEC Reference Data ================== EdgarTools ships with a comprehensive set of SEC reference data — ticker-to-CIK mappings, exchange listings, industry classifications, CUSIP lookups, place codes, and form descriptions. Most of this data is **bundled with the package** and works offline with zero configuration. What's Included --------------- | Data | Source | Network? | Function / Location | | --- | --- | --- | --- | | **~10,600 tickers with CIK and exchange** | Bundled parquet | No | `Company("AAPL")`, `find_cik("AAPL")` | | **CUSIP-to-ticker mapping** | Bundled parquet | No | `cusip_ticker_mapping()`, `get_ticker_from_cusip()` | | **SEC form descriptions** | Bundled CSV | No | `describe_form("10-K")` | | **Place codes (states/countries)** | Bundled CSV | No | `get_place_name()`, `get_filer_type()` | | **Popular stock lists** | Bundled CSV | No | `get_popular_companies()`, `get_faang_companies()` | | **Full SEC ticker universe** | SEC API / local download | Yes (once) | `download_edgar_data(reference=True)` | All bundled data lives in `edgar/reference/data/` inside the installed package and is loaded automatically. Ticker-to-CIK Resolution ------------------------ The most common use of reference data is resolving ticker symbols to SEC CIK numbers. This happens automatically when you call `Company()`. ### How It Works When you call `Company("AAPL")`, edgartools resolves the ticker using a three-level waterfall: 1. **Bundled parquet** (instant, no network) — ~10,600 exchange-listed tickers ship with the package 2. **Local downloaded data** (if configured) — the full SEC ticker universe including recent IPOs 3. **Live SEC API** (fallback) — fetched once per session and cached in memory This means **`Company()` lookups work offline by default** for established tickers. `from edgar import Company # Works offline — no internet needed company = Company("AAPL") print(f"{company.name} (CIK: {company.cik})")` ### Lightweight CIK Lookup If you just need the CIK number without loading the full company object: `from edgar.reference.tickers import find_cik cik = find_cik("NVDA") print(f"NVDA CIK: {cik}") # 1045810` ### What's in the Bundled Data `from edgar.reference.tickers import load_company_tickers_from_package bundled = load_company_tickers_from_package() print(f"Tickers: {len(bundled):,}") print(f"Columns: {list(bundled.columns)}") print(f"Exchanges: {sorted(bundled['exchange'].dropna().unique())}")` Output: `Tickers: 10,652 Columns: ['cik', 'ticker', 'company', 'exchange'] Exchanges: ['CBOE', 'NYSE', 'Nasdaq', 'OTC']` Companies by Exchange --------------------- Get all companies listed on a specific stock exchange: `from edgar.reference import get_companies_by_exchanges # Single exchange nyse = get_companies_by_exchanges("NYSE") print(f"NYSE companies: {len(nyse):,}") # Multiple exchanges major = get_companies_by_exchanges(["NYSE", "Nasdaq"]) print(f"NYSE + Nasdaq: {len(major):,}")` Returns a DataFrame with columns `[cik, ticker, name, exchange]`. Companies by Industry --------------------- The SEC classifies companies using SIC (Standard Industrial Classification) codes: `from edgar.reference import get_companies_by_industry # Software companies (SIC 7372) software = get_companies_by_industry(sic=7372) print(f"Software companies: {len(software)}")` ### Industry Convenience Functions Common industries have dedicated functions: `from edgar.reference import ( get_banking_companies, get_pharmaceutical_companies, get_software_companies, get_semiconductor_companies, get_oil_gas_companies, get_real_estate_companies, get_insurance_companies, get_retail_companies, get_biotechnology_companies, get_investment_companies, ) banks = get_banking_companies() pharma = get_pharmaceutical_companies()` Companies by State ------------------ Find companies by their state of incorporation: `from edgar.reference import get_companies_by_state delaware = get_companies_by_state("DE") print(f"Delaware companies: {len(delaware):,}") california = get_companies_by_state("CA") print(f"California companies: {len(california):,}")` Popular Company Lists --------------------- Curated lists useful for demos, testing, and quick analysis: `from edgar.reference import ( get_popular_companies, get_faang_companies, get_tech_giants, get_dow_jones_sample, PopularityTier, ) # All popular companies popular = get_popular_companies() # By popularity tier mega_cap = get_popular_companies(PopularityTier.MEGA_CAP) # Top 10 top_50 = get_popular_companies(PopularityTier.POPULAR) # Top 50 mainstream = get_popular_companies(PopularityTier.MAINSTREAM) # Top 100 # Named groups faang = get_faang_companies() # Meta, Apple, Amazon, Netflix, Google tech = get_tech_giants() # Major tech companies dow = get_dow_jones_sample() # Dow Jones sample` Form Descriptions ----------------- Look up what any SEC form type means: `from edgar.reference import describe_form print(describe_form("10-K")) # Form 10-K: Annual report for public companies print(describe_form("DEF 14A")) # Form DEF 14A: Definitive proxy statement print(describe_form("SC 13D")) # Form SC 13D: Beneficial ownership report (>5%) # Without the "Form" prefix print(describe_form("8-K", prepend_form=False)) # Current report` CUSIP-to-Ticker Mapping ----------------------- A CUSIP is a 9-character identifier used in securities trading. EdgarTools includes a CUSIP-to-ticker mapping, primarily used by the 13F institutional holdings parser: `from edgar.reference import cusip_ticker_mapping, get_ticker_from_cusip # Single lookup ticker = get_ticker_from_cusip("037833100") # Apple's CUSIP print(f"037833100 -> {ticker}") # AAPL # Full mapping as DataFrame (indexed by CUSIP) mapping = cusip_ticker_mapping() print(f"Total CUSIP mappings: {len(mapping):,}")` Place Codes and Filer Classification ------------------------------------ The SEC uses internal codes for states and countries. EdgarTools decodes these and classifies filers: `from edgar.reference import ( get_place_name, get_filer_type, is_us_company, is_foreign_company, is_canadian_company, ) # Decode place codes get_place_name("DE") # "Delaware" get_place_name("X0") # "United Kingdom" get_place_name("A6") # "Alberta, Canada" # Classify filer type get_filer_type("DE") # "Domestic" get_filer_type("X0") # "Foreign" get_filer_type("A6") # "Canadian" # Boolean checks is_us_company("DE") # True is_foreign_company("X0") # True is_canadian_company("A6") # True` Building Research Datasets -------------------------- The `CompanySubset` class provides a fluent interface for building precise company selections: `from edgar.reference import CompanySubset # 50 random NYSE/Nasdaq companies (reproducible) research_set = (CompanySubset() .from_exchange(["NYSE", "Nasdaq"]) .sample(50, random_state=42) .get()) print(f"Research set: {len(research_set)} companies")` ### Stratified Sampling Maintain exchange proportions in your sample: `from edgar.reference import get_stratified_sample sample = get_stratified_sample(n=100, stratify_by="exchange", random_state=42) print(sample["exchange"].value_counts(normalize=True))` ### Filtering and Combining `from edgar.reference import ( filter_companies, exclude_companies, combine_company_sets, intersect_company_sets, get_all_companies, ) all_companies = get_all_companies() # Include specific tickers faang = filter_companies(all_companies, ticker_list=["META", "AAPL", "AMZN", "NFLX", "GOOGL"]) # Exclude specific companies non_tech = exclude_companies(all_companies, ticker_list=["AAPL", "MSFT", "GOOGL"]) # Set operations nyse = get_companies_by_exchanges("NYSE") popular = get_popular_companies() nyse_popular = intersect_company_sets([nyse, popular])` For the full CompanySubset API, see the [Company Subsets](https://edgartools.readthedocs.io/en/stable/company-subsets/) guide. Offline Setup ------------- The bundled parquet covers ~10,600 exchange-listed tickers. For the **full SEC ticker universe** — including recent IPOs, mutual funds, and non-exchange filers — download reference data locally: `from edgar import download_edgar_data, use_local_storage # One-time download (~50 MB, takes a few seconds) download_edgar_data(submissions=False, facts=False, reference=True) # Enable local storage use_local_storage() # Now all lookups use the complete local data company = Company("AAPL")` What gets downloaded | File | Content | | --- | --- | | `company_tickers.json` | All SEC-registered tickers with CIK | | `company_tickers_exchange.json` | Tickers with exchange information | | `mutual_fund_tickers.json` | Mutual fund ticker-to-CIK mappings | Downloaded files are stored in `~/.edgar/reference/` (or your configured data directory). For full offline capabilities beyond reference data (submissions, facts, filing documents), see the [Local Storage](https://edgartools.readthedocs.io/en/stable/guides/local-storage/) guide. Quick Reference --------------- `from edgar import Company, download_edgar_data, use_local_storage from edgar.reference import * from edgar.reference.tickers import find_cik # ── Ticker / CIK ── Company("AAPL") # Ticker -> full company object find_cik("AAPL") # Ticker -> CIK only (lightweight) # ── By exchange ── get_companies_by_exchanges("NYSE") # Single exchange get_companies_by_exchanges(["NYSE", "Nasdaq"]) # Multiple # ── By industry / state ── get_companies_by_industry(sic=7372) # By SIC code get_companies_by_state("DE") # By incorporation state # ── Popular companies ── get_popular_companies() # All popular get_faang_companies() # FAANG get_tech_giants() # Major tech get_banking_companies() # Banks (+ pharma, software, etc.) # ── Form descriptions ── describe_form("10-K") # "Form 10-K: Annual report..." # ── CUSIP ── get_ticker_from_cusip("037833100") # CUSIP -> ticker cusip_ticker_mapping() # Full mapping DataFrame # ── Place codes ── get_place_name("DE") # "Delaware" get_filer_type("DE") # "Domestic" is_us_company("DE") # True # ── Research datasets ── CompanySubset().from_exchange("NYSE").sample(100).get() get_stratified_sample(n=100, stratify_by="exchange") get_random_sample(n=50) # ── Full offline setup ── download_edgar_data(submissions=False, facts=False, reference=True) use_local_storage()` Related ------- * **[Find a Company](https://edgartools.readthedocs.io/en/stable/guides/finding-companies/) ** — Ticker lookup, CIK, name search, screening * **[Company Subsets](https://edgartools.readthedocs.io/en/stable/company-subsets/) ** — Full CompanySubset API and advanced dataset creation * **[Company Classification](https://edgartools.readthedocs.io/en/stable/guides/company-classification/) ** — Filer types, business categories, SIC codes * **[Local Storage](https://edgartools.readthedocs.io/en/stable/guides/local-storage/) ** — Full offline setup for submissions, facts, and filings Back to top --- # Reference Data - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/reference-data/#sec-reference-data) SEC Reference Data ================== EdgarTools ships with a comprehensive set of SEC reference data — ticker-to-CIK mappings, exchange listings, industry classifications, CUSIP lookups, place codes, and form descriptions. Most of this data is **bundled with the package** and works offline with zero configuration. What's Included --------------- | Data | Source | Network? | Function / Location | | --- | --- | --- | --- | | **~10,600 tickers with CIK and exchange** | Bundled parquet | No | `Company("AAPL")`, `find_cik("AAPL")` | | **CUSIP-to-ticker mapping** | Bundled parquet | No | `cusip_ticker_mapping()`, `get_ticker_from_cusip()` | | **SEC form descriptions** | Bundled CSV | No | `describe_form("10-K")` | | **Place codes (states/countries)** | Bundled CSV | No | `get_place_name()`, `get_filer_type()` | | **Popular stock lists** | Bundled CSV | No | `get_popular_companies()`, `get_faang_companies()` | | **Full SEC ticker universe** | SEC API / local download | Yes (once) | `download_edgar_data(reference=True)` | All bundled data lives in `edgar/reference/data/` inside the installed package and is loaded automatically. Ticker-to-CIK Resolution ------------------------ The most common use of reference data is resolving ticker symbols to SEC CIK numbers. This happens automatically when you call `Company()`. ### How It Works When you call `Company("AAPL")`, edgartools resolves the ticker using a three-level waterfall: 1. **Bundled parquet** (instant, no network) — ~10,600 exchange-listed tickers ship with the package 2. **Local downloaded data** (if configured) — the full SEC ticker universe including recent IPOs 3. **Live SEC API** (fallback) — fetched once per session and cached in memory This means **`Company()` lookups work offline by default** for established tickers. `from edgar import Company # Works offline — no internet needed company = Company("AAPL") print(f"{company.name} (CIK: {company.cik})")` ### Lightweight CIK Lookup If you just need the CIK number without loading the full company object: `from edgar.reference.tickers import find_cik cik = find_cik("NVDA") print(f"NVDA CIK: {cik}") # 1045810` ### What's in the Bundled Data `from edgar.reference.tickers import load_company_tickers_from_package bundled = load_company_tickers_from_package() print(f"Tickers: {len(bundled):,}") print(f"Columns: {list(bundled.columns)}") print(f"Exchanges: {sorted(bundled['exchange'].dropna().unique())}")` Output: `Tickers: 10,652 Columns: ['cik', 'ticker', 'company', 'exchange'] Exchanges: ['CBOE', 'NYSE', 'Nasdaq', 'OTC']` Companies by Exchange --------------------- Get all companies listed on a specific stock exchange: `from edgar.reference import get_companies_by_exchanges # Single exchange nyse = get_companies_by_exchanges("NYSE") print(f"NYSE companies: {len(nyse):,}") # Multiple exchanges major = get_companies_by_exchanges(["NYSE", "Nasdaq"]) print(f"NYSE + Nasdaq: {len(major):,}")` Returns a DataFrame with columns `[cik, ticker, name, exchange]`. Companies by Industry --------------------- The SEC classifies companies using SIC (Standard Industrial Classification) codes: `from edgar.reference import get_companies_by_industry # Software companies (SIC 7372) software = get_companies_by_industry(sic=7372) print(f"Software companies: {len(software)}")` ### Industry Convenience Functions Common industries have dedicated functions: `from edgar.reference import ( get_banking_companies, get_pharmaceutical_companies, get_software_companies, get_semiconductor_companies, get_oil_gas_companies, get_real_estate_companies, get_insurance_companies, get_retail_companies, get_biotechnology_companies, get_investment_companies, ) banks = get_banking_companies() pharma = get_pharmaceutical_companies()` Companies by State ------------------ Find companies by their state of incorporation: `from edgar.reference import get_companies_by_state delaware = get_companies_by_state("DE") print(f"Delaware companies: {len(delaware):,}") california = get_companies_by_state("CA") print(f"California companies: {len(california):,}")` Popular Company Lists --------------------- Curated lists useful for demos, testing, and quick analysis: `from edgar.reference import ( get_popular_companies, get_faang_companies, get_tech_giants, get_dow_jones_sample, PopularityTier, ) # All popular companies popular = get_popular_companies() # By popularity tier mega_cap = get_popular_companies(PopularityTier.MEGA_CAP) # Top 10 top_50 = get_popular_companies(PopularityTier.POPULAR) # Top 50 mainstream = get_popular_companies(PopularityTier.MAINSTREAM) # Top 100 # Named groups faang = get_faang_companies() # Meta, Apple, Amazon, Netflix, Google tech = get_tech_giants() # Major tech companies dow = get_dow_jones_sample() # Dow Jones sample` Form Descriptions ----------------- Look up what any SEC form type means: `from edgar.reference import describe_form print(describe_form("10-K")) # Form 10-K: Annual report for public companies print(describe_form("DEF 14A")) # Form DEF 14A: Definitive proxy statement print(describe_form("SC 13D")) # Form SC 13D: Beneficial ownership report (>5%) # Without the "Form" prefix print(describe_form("8-K", prepend_form=False)) # Current report` CUSIP-to-Ticker Mapping ----------------------- A CUSIP is a 9-character identifier used in securities trading. EdgarTools includes a CUSIP-to-ticker mapping, primarily used by the 13F institutional holdings parser: `from edgar.reference import cusip_ticker_mapping, get_ticker_from_cusip # Single lookup ticker = get_ticker_from_cusip("037833100") # Apple's CUSIP print(f"037833100 -> {ticker}") # AAPL # Full mapping as DataFrame (indexed by CUSIP) mapping = cusip_ticker_mapping() print(f"Total CUSIP mappings: {len(mapping):,}")` Place Codes and Filer Classification ------------------------------------ The SEC uses internal codes for states and countries. EdgarTools decodes these and classifies filers: `from edgar.reference import ( get_place_name, get_filer_type, is_us_company, is_foreign_company, is_canadian_company, ) # Decode place codes get_place_name("DE") # "Delaware" get_place_name("X0") # "United Kingdom" get_place_name("A6") # "Alberta, Canada" # Classify filer type get_filer_type("DE") # "Domestic" get_filer_type("X0") # "Foreign" get_filer_type("A6") # "Canadian" # Boolean checks is_us_company("DE") # True is_foreign_company("X0") # True is_canadian_company("A6") # True` Building Research Datasets -------------------------- The `CompanySubset` class provides a fluent interface for building precise company selections: `from edgar.reference import CompanySubset # 50 random NYSE/Nasdaq companies (reproducible) research_set = (CompanySubset() .from_exchange(["NYSE", "Nasdaq"]) .sample(50, random_state=42) .get()) print(f"Research set: {len(research_set)} companies")` ### Stratified Sampling Maintain exchange proportions in your sample: `from edgar.reference import get_stratified_sample sample = get_stratified_sample(n=100, stratify_by="exchange", random_state=42) print(sample["exchange"].value_counts(normalize=True))` ### Filtering and Combining `from edgar.reference import ( filter_companies, exclude_companies, combine_company_sets, intersect_company_sets, get_all_companies, ) all_companies = get_all_companies() # Include specific tickers faang = filter_companies(all_companies, ticker_list=["META", "AAPL", "AMZN", "NFLX", "GOOGL"]) # Exclude specific companies non_tech = exclude_companies(all_companies, ticker_list=["AAPL", "MSFT", "GOOGL"]) # Set operations nyse = get_companies_by_exchanges("NYSE") popular = get_popular_companies() nyse_popular = intersect_company_sets([nyse, popular])` For the full CompanySubset API, see the [Company Subsets](https://edgartools.readthedocs.io/en/latest/company-subsets/) guide. Offline Setup ------------- The bundled parquet covers ~10,600 exchange-listed tickers. For the **full SEC ticker universe** — including recent IPOs, mutual funds, and non-exchange filers — download reference data locally: `from edgar import download_edgar_data, use_local_storage # One-time download (~50 MB, takes a few seconds) download_edgar_data(submissions=False, facts=False, reference=True) # Enable local storage use_local_storage() # Now all lookups use the complete local data company = Company("AAPL")` What gets downloaded | File | Content | | --- | --- | | `company_tickers.json` | All SEC-registered tickers with CIK | | `company_tickers_exchange.json` | Tickers with exchange information | | `mutual_fund_tickers.json` | Mutual fund ticker-to-CIK mappings | Downloaded files are stored in `~/.edgar/reference/` (or your configured data directory). For full offline capabilities beyond reference data (submissions, facts, filing documents), see the [Local Storage](https://edgartools.readthedocs.io/en/latest/guides/local-storage/) guide. Quick Reference --------------- `from edgar import Company, download_edgar_data, use_local_storage from edgar.reference import * from edgar.reference.tickers import find_cik # ── Ticker / CIK ── Company("AAPL") # Ticker -> full company object find_cik("AAPL") # Ticker -> CIK only (lightweight) # ── By exchange ── get_companies_by_exchanges("NYSE") # Single exchange get_companies_by_exchanges(["NYSE", "Nasdaq"]) # Multiple # ── By industry / state ── get_companies_by_industry(sic=7372) # By SIC code get_companies_by_state("DE") # By incorporation state # ── Popular companies ── get_popular_companies() # All popular get_faang_companies() # FAANG get_tech_giants() # Major tech get_banking_companies() # Banks (+ pharma, software, etc.) # ── Form descriptions ── describe_form("10-K") # "Form 10-K: Annual report..." # ── CUSIP ── get_ticker_from_cusip("037833100") # CUSIP -> ticker cusip_ticker_mapping() # Full mapping DataFrame # ── Place codes ── get_place_name("DE") # "Delaware" get_filer_type("DE") # "Domestic" is_us_company("DE") # True # ── Research datasets ── CompanySubset().from_exchange("NYSE").sample(100).get() get_stratified_sample(n=100, stratify_by="exchange") get_random_sample(n=50) # ── Full offline setup ── download_edgar_data(submissions=False, facts=False, reference=True) use_local_storage()` Related ------- * **[Find a Company](https://edgartools.readthedocs.io/en/latest/guides/finding-companies/) ** — Ticker lookup, CIK, name search, screening * **[Company Subsets](https://edgartools.readthedocs.io/en/latest/company-subsets/) ** — Full CompanySubset API and advanced dataset creation * **[Company Classification](https://edgartools.readthedocs.io/en/latest/guides/company-classification/) ** — Filer types, business categories, SIC codes * **[Local Storage](https://edgartools.readthedocs.io/en/latest/guides/local-storage/) ** — Full offline setup for submissions, facts, and filings Back to top --- # Overview - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/data-objects/#sec-filing-data-objects-parsed-python-objects-for-every-form-type) SEC Filing Data Objects: Parsed Python Objects for Every Form Type ================================================================== Every SEC filing can be parsed into a structured Python object with one call: `filing.obj() # returns a TenK, EightK, ThirteenF, etc.` Browse the filing types below to find what you need. * * * Fund Entities ------------- Look up mutual funds and ETFs by ticker, series ID, or CIK. Navigate fund hierarchies and access portfolio reports. `from edgar import Fund, find_funds fund = Fund("VFINX") # Ticker, series ID, or CIK fund.get_portfolio() # Latest portfolio holdings` [Fund Entities guide](https://edgartools.readthedocs.io/en/latest/guides/fund-entity-guide/) * * * Annual & Quarterly Reports (10-K / 10-Q) ---------------------------------------- Read a company's financials, risk factors, and business description. `tenk = filing.obj() # TenK or TenQ tenk.income_statement # formatted financial statement tenk.risk_factors # full section text tenk.auditor # auditor name, PCAOB ID, location tenk.subsidiaries # subsidiaries from Exhibit 21 (10-K only) tenk.reports # XBRL viewer pages (statements, notes, details)` [Annual & Quarterly Reports](https://edgartools.readthedocs.io/en/latest/concepts/data-objects/) * * * Current Events (8-K) -------------------- Find out what just happened -- acquisitions, officer changes, earnings releases. `eightk = filing.obj() # EightK eightk.items # list of reported event codes eightk.press_releases # attached press releases` [Current Events guide](https://edgartools.readthedocs.io/en/latest/data-objects/guides/eightk-data-object-guide.md) * * * Insider Trades (Form 4) ----------------------- See who bought or sold shares and at what price. `form4 = filing.obj() # Ownership form4.reporting_owner # insider name form4.transactions # buy/sell details with prices` [Insider Trades guide](https://edgartools.readthedocs.io/en/latest/insider-filings/) See it live on edgar.tools Every filing type above — 10-K, 8-K, Form 4, 13F, proxy statements — is also browsable on **edgar.tools** with AI enrichment layered on top: * **[Browse Apple's filings, financials, and insider trades →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=data-objects) ** * **[Watch filings arrive in real time →](https://app.edgar.tools/filings?utm_source=edgartools-docs&utm_medium=see-live&utm_content=data-objects) ** * **[Search disclosures across 12 XBRL topics →](https://app.edgar.tools/disclosures?utm_source=edgartools-docs&utm_medium=see-live&utm_content=data-objects) ** Includes AI-classified 8-K events, insider sentiment analysis, and multi-year disclosure comparison. Free tier available. * * * Beneficial Ownership (Schedule 13D/G) ------------------------------------- Track activist investors and large institutional holders who own 5%+ of a company. `schedule = filing.obj() # Schedule13D or Schedule13G schedule.total_shares # aggregate beneficial ownership schedule.items.item4_purpose_of_transaction # activist intent (13D only)` [Beneficial Ownership guide](https://edgartools.readthedocs.io/en/latest/guides/schedule13dg-data-object-guide/) * * * Institutional Portfolios (13F) ------------------------------ Explore hedge fund and institutional investor holdings. `thirteenf = filing.obj() # ThirteenF thirteenf.infotable # full holdings table thirteenf.total_value # portfolio market value` [Institutional Portfolios guide](https://edgartools.readthedocs.io/en/latest/guides/thirteenf-data-object-guide/) * * * Proxy & Governance (DEF 14A) ---------------------------- Review executive compensation, board nominees, and shareholder proposals. `proxy = filing.obj() # ProxyStatement proxy.executive_compensation # pay tables proxy.proposals # shareholder vote items` [Proxy & Governance guide](https://edgartools.readthedocs.io/en/latest/guides/proxystatement-data-object-guide/) * * * Private Offerings (Form D) -------------------------- Track exempt securities offerings and the companies raising capital. `formd = filing.obj() # FormD formd.offering # offering details and amounts formd.recipients # related persons` [Private Offerings guide](https://edgartools.readthedocs.io/en/latest/guides/formd-data-object-guide/) * * * Crowdfunding Offerings (Form C) ------------------------------- Monitor crowdfunding campaigns under Regulation CF, including offering terms and issuer financials. `formc = filing.obj() # FormC formc.offering_information # target amount, deadline, securities formc.annual_report_disclosure # issuer financials (if C-AR)` [Crowdfunding guide](https://edgartools.readthedocs.io/en/latest/guides/formc-data-object-guide/) * * * Insider Sale Notices (Form 144) ------------------------------- Monitor planned insider sales before they happen. `form144 = filing.obj() # Form144 form144.proposed_sale_amount # shares to be sold form144.securities # security details` [Insider Sale Notices guide](https://edgartools.readthedocs.io/en/latest/guides/form144-data-object-guide/) * * * Fund Shareholder Reports (N-CSR / N-CSRS) ----------------------------------------- Parse certified annual and semiannual shareholder reports with expense ratios, performance data, and share class details. `report = filing.obj() # FundShareholderReport report.expense_data() # expense ratios per share class report.performance_data() # annual returns per share class` [Fund Shareholder Reports guide](https://edgartools.readthedocs.io/en/latest/guides/fundshareholderreport-data-object-guide/) * * * Fund Portfolio Holdings (NPORT-P) --------------------------------- Parse monthly mutual fund and ETF portfolio holdings -- every stock, bond, and derivative position. `report = filing.obj() # FundReport report.investment_data() # All portfolio positions as DataFrame` [Fund Portfolio Holdings guide](https://edgartools.readthedocs.io/en/latest/guides/nport-data-object-guide/) * * * Money Market Funds (N-MFP) -------------------------- Parse money market fund filings with portfolio holdings, yields, NAV, and liquidity metrics. `mmf = filing.obj() # MoneyMarketFund mmf.portfolio_data() # Securities sorted by market value` [Money Market Funds guide](https://edgartools.readthedocs.io/en/latest/guides/moneymarketfund-data-object-guide/) * * * Fund Census (N-CEN) ------------------- Parse annual fund census filings with series data, service providers, and ETF details. `census = filing.obj() # FundCensus census.series_data() # Fund series summary` [Fund Census guide](https://edgartools.readthedocs.io/en/latest/guides/fundcensus-data-object-guide/) * * * Fund Voting Records (N-PX) -------------------------- See how mutual funds voted on shareholder proposals. `npx = filing.obj() # FundReport npx.votes # vote records by proposal` [Fund Voting Records guide](https://edgartools.readthedocs.io/en/latest/guides/npx-data-object-guide/) * * * ABS Distribution Reports (Form 10-D) ------------------------------------ Extract structured CMBS loan and property data from asset-backed securities distribution reports. `ten_d = filing.obj() # TenD (CMBS only) ten_d.loans # loan-level DataFrame ten_d.properties # property-level DataFrame ten_d.asset_data.summary() # pool statistics` [ABS Distribution Reports guide](https://edgartools.readthedocs.io/en/latest/guides/tend-data-object-guide/) * * * Municipal Advisors (MA-I) ------------------------- Look up municipal advisor registrations and disciplinary history. `mai = filing.obj() # MunicipalAdvisorForm mai.advisor_name # advisor details` [Municipal Advisors guide](https://edgartools.readthedocs.io/en/latest/guides/mai-data-object-guide/) * * * Prospectus Supplements (424B) ----------------------------- Extract offering terms, pricing, underwriting, and dilution from shelf takedown prospectuses. `prospectus = filing.obj() # Prospectus424B deal = prospectus.deal # Deal: normalized deal summary deal.price # per-share price (float) deal.gross_proceeds # total offering amount deal.discount_rate # underwriting fee as fraction of price` [Prospectus Supplements guide](https://edgartools.readthedocs.io/en/latest/guides/prospectus424b-data-object-guide/) * * * How it works ------------ Call `filing.obj()` on any supported filing. EdgarTools detects the form type, parses the raw HTML/XML/XBRL, and returns the right data object. If a filing type isn't supported yet, you'll get an `UnsupportedFilingTypeError`. `from edgar import Company apple = Company("AAPL") filing = apple.get_latest_filing("10-K") tenk = filing.obj() # returns a TenK with all sections and financials` Back to top --- # Company Classification - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/company-classification/#company-classification) Company Classification ====================== EdgarTools automatically classifies every SEC entity across multiple dimensions: whether it is a domestic or foreign registrant, what kind of business it operates, and its regulatory filing status. These properties are derived from SEC data — SIC codes, state of incorporation, and filing history — so you rarely need to look anything up manually. Filer Type: Domestic, Foreign, or Canadian ------------------------------------------ The `filer_type` property tells you where a company is incorporated. This matters for understanding which annual report form the company files: domestic companies file 10-K, foreign private issuers file 20-F, and Canadian issuers file 40-F. `from edgar import Company Company("AAPL").filer_type # 'Domestic' Company("BABA").filer_type # 'Foreign' Company("CNQ").filer_type # 'Canadian'` The `is_foreign` convenience property returns `True` for both Foreign and Canadian filers: `Company("BABA").is_foreign # True Company("CNQ").is_foreign # True Company("AAPL").is_foreign # False` ### How filer type is determined EdgarTools uses a two-stage approach: 1. **State of incorporation** (preferred): The SEC stores a state or country code for each registered entity. A US state code means domestic; a country code from outside Canada means foreign; Canada codes mean Canadian. 2. **Filing history fallback**: When the state of incorporation is absent, EdgarTools inspects the entity's recent filings. A 40-F signals Canadian; a 20-F or 6-K signals foreign; a 10-K or 10-Q signals domestic. Extended fallbacks cover ADR deposit registrations (`F-6`), foreign registration statements (`F-1`, `F-3`), and domestic-only forms like Regulation Crowdfunding (`C`). Business Category ----------------- The `business_category` property classifies what kind of entity a company is. This is useful when building screens or analysis pipelines that should behave differently for, say, a bank versus a REIT versus an ordinary operating company. `from edgar import Company Company("AAPL").business_category # 'Operating Company' Company("AGNC").business_category # 'REIT' Company("JPM").business_category # 'Bank' Company("MET").business_category # 'Insurance Company' Company("ARCC").business_category # 'BDC'` ### Available categories | Category | Description | | --- | --- | | `Operating Company` | Standard corporation — the default for most SEC filers | | `REIT` | Real Estate Investment Trust (SIC 6798) | | `Bank` | Commercial banks and savings institutions | | `Insurance Company` | Life, casualty, title, and similar insurers | | `ETF` | Exchange-traded fund | | `Mutual Fund` | Open-end registered investment company | | `Closed-End Fund` | Closed-end registered investment company | | `BDC` | Business Development Company | | `Investment Manager` | Asset manager or institutional investment adviser | | `Holding Company` | Pure holding company (SIC 6719) | | `SPAC` | Blank check / special purpose acquisition company | | `Unknown` | Insufficient signals for classification | ### Convenience predicates Three boolean methods let you check the broad category without pattern-matching strings: `company = Company("AAPL") company.is_operating_company() # True company.is_fund() # False company.is_financial_institution() # False` `company = Company("JPM") company.is_operating_company() # False company.is_financial_institution() # True (Banks, Insurance, Investment Managers, BDCs)` `company = Company("SPY") company.is_fund() # True (ETF, Mutual Fund, Closed-End Fund)` ### How business category is determined Classification uses a priority chain: 1. **Definitive SIC codes**: SIC 6798 → REIT; SIC 6770 → SPAC; SIC 6021–6036 → Bank; SIC 6311–6371 → Insurance Company. 2. **Investment company forms**: Primary investment forms (`N-CSR`, `NPORT-P`) trigger fund classification; the name and entity type then distinguish ETF from Mutual Fund from Closed-End Fund. 3. **BDC signals**: Operating entities that file `N-2` forms or whose names contain "Capital Corp". 4. **Investment manager signals**: Entities with SIC 6211 or 6282, or that file `13F-HR`. 5. **Holding company**: SIC 6719. 6. **Default**: Operating Company. Filer Category: SEC Accelerated Filer Status -------------------------------------------- The SEC requires companies above certain public float thresholds to file on accelerated timelines. The `filer_category` property captures this classification. `from edgar import Company apple = Company("AAPL") apple.is_large_accelerated_filer # True (public float >= $700M) apple.is_accelerated_filer # False apple.is_smaller_reporting_company # False apple.is_emerging_growth_company # False` For smaller companies: `# A hypothetical small-cap company company = Company("BYFC") company.is_non_accelerated_filer # True (public float < $75M) company.is_smaller_reporting_company # True (public float < $250M or revenue < $100M)` ### Filer status thresholds | Status | Public Float | | --- | --- | | Large Accelerated Filer | \>= $700 million | | Accelerated Filer | \>= $75 million and < $700 million | | Non-Accelerated Filer | < $75 million | Two additional qualifications may apply alongside any base status: * **Smaller Reporting Company (SRC)**: Public float below $250 million, or annual revenue below $100 million with no public float above $700 million. SRCs may use scaled disclosure requirements. * **Emerging Growth Company (EGC)**: Revenue below $1.235 billion and IPO within the past five years. EGCs may defer certain accounting standards. For the full `FilerCategory` object with enum access: `from edgar import Company from edgar.enums import FilerStatus, FilerCategory category = Company("AAPL").filer_category category.status # FilerStatus.LARGE_ACCELERATED str(category) # 'Large accelerated filer' category.qualifications # [] category.is_smaller_reporting_company # False` Industry: SIC Code and Description ---------------------------------- Every SEC registrant is assigned a Standard Industrial Classification (SIC) code. EdgarTools exposes both the code and its human-readable description: `from edgar import Company apple = Company("AAPL") apple.sic # 3571 apple.industry # 'Electronic Computers' jpm = Company("JPM") jpm.sic # 6022 jpm.industry # 'State commercial banks-Federal Reserve members & state (non members)'` Entity vs. Individual --------------------- Not every SEC filer is a company. Insiders and beneficial owners file ownership forms (Forms 3, 4, 5 and Schedule 13D/G) as individuals. EdgarTools distinguishes these automatically: `from edgar import Company Company("AAPL").is_company # True Company("AAPL").is_individual # False` When you load an entity by CIK and that entity turns out to be a person rather than a company, `is_individual` returns `True`. This typically happens when looking up a CIK obtained from an ownership filing. Classification uses a nine-signal priority chain: exchange listings, state of incorporation, entity type from SEC data, filing history, EIN, and name keywords. Companies with tickers or a state of incorporation are definitively classified as companies. Filers with only insider ownership forms in their history are classified as individuals. Quick Reference --------------- | Property | Type | Returns | | --- | --- | --- | | `filer_type` | `str \\| None` | `'Domestic'`, `'Foreign'`, `'Canadian'`, or `None` | | `is_foreign` | `bool` | `True` for Foreign or Canadian registrants | | `business_category` | `str` | See business category table above | | `is_operating_company()` | `bool` | `True` for standard operating companies | | `is_fund()` | `bool` | `True` for ETF, Mutual Fund, or Closed-End Fund | | `is_financial_institution()` | `bool` | `True` for Bank, Insurance, Investment Manager, or BDC | | `sic` | `int \\| None` | Standard Industrial Classification code | | `industry` | `str \\| None` | SIC description | | `filer_category` | `FilerCategory` | Full parsed filer category object | | `is_large_accelerated_filer` | `bool` | Public float >= $700M | | `is_accelerated_filer` | `bool` | Public float >= $75M and < $700M | | `is_non_accelerated_filer` | `bool` | Public float < $75M | | `is_smaller_reporting_company` | `bool` | Qualifies as SRC | | `is_emerging_growth_company` | `bool` | Qualifies as EGC | | `is_company` | `bool` | `True` if the filer is a company | | `is_individual` | `bool` | `True` if the filer is a person | Related Guides -------------- * [Finding Companies](https://edgartools.readthedocs.io/en/latest/guides/finding-companies/) — Look up companies by ticker, CIK, or name * [Entity API Guide](https://edgartools.readthedocs.io/en/latest/guides/entity-api-guide/) — Filer category details and company icons * [BDC Guide](https://edgartools.readthedocs.io/en/latest/guides/bdc-guide/) — Working with Business Development Companies * [Fund Entity Guide](https://edgartools.readthedocs.io/en/latest/guides/fund-entity-guide/) — ETFs, mutual funds, and closed-end funds Back to top --- # Business Development Companies - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/bdc-guide/#business-development-companies) Business Development Companies ============================== Access and analyze Business Development Companies (BDCs) - specialized investment companies that invest in small and mid-sized private companies. What Are BDCs? -------------- Business Development Companies are closed-end investment funds that: * Invest in small and mid-sized private U.S. companies * Provide financing (loans) and equity investments * Must distribute 90%+ of taxable income as dividends * File with the SEC under file numbers starting with "814-" BDCs provide a unique window into private credit markets through their quarterly Schedule of Investments disclosures. Finding BDCs ------------ ### List All BDCs Get the complete list of ~196 BDCs from the SEC BDC Report: `from edgar.bdc import get_bdc_list bdcs = get_bdc_list() print(f"Found {len(bdcs)} BDCs")` **Output:** `Found 196 BDCs` ### Get a Specific BDC Find a BDC by ticker or CIK: `from edgar.bdc import get_bdc_list bdcs = get_bdc_list() # By ticker arcc = bdcs.get_by_ticker("ARCC") print(arcc.name) # ARES CAPITAL CORP # By CIK main = bdcs.get_by_cik(1396440) print(main.name) # MAIN STREET CAPITAL CORP` ### Search for BDCs Use fuzzy search to find BDCs by name or ticker: `from edgar.bdc import find_bdc # Search by name results = find_bdc("Ares") print(results[0].name) # ARES CAPITAL CORP # Search by ticker results = find_bdc("MAIN") print(results[0].name) # MAIN STREET CAPITAL CORP` **Key points:** - Search is case-insensitive - Supports partial name matching - Returns ranked results by relevance ### Filter BDCs Filter by state or activity status: `bdcs = get_bdc_list() # Active BDCs only (filed within last 18 months) active_bdcs = bdcs.filter(active=True) print(f"Active BDCs: {len(active_bdcs)}") # BDCs in New York ny_bdcs = bdcs.filter(state='NY') print(f"NY-based BDCs: {len(ny_bdcs)}") # Combine filters ny_active = bdcs.filter(state='NY', active=True)` ### Check if a Company is a BDC Verify whether a CIK belongs to a BDC: `from edgar.bdc import is_bdc_cik is_bdc_cik(1287750) # True - ARCC is a BDC is_bdc_cik(320193) # False - Apple is not a BDC` BDC Properties -------------- Each BDC entity provides useful information: `arcc = bdcs.get_by_ticker("ARCC") # Basic info print(f"Name: {arcc.name}") print(f"CIK: {arcc.cik}") print(f"File Number: {arcc.file_number}") # 814-00663 print(f"State: {arcc.state}") print(f"Active: {arcc.is_active}") # Filing info print(f"Last Filing: {arcc.last_filing_date}") print(f"Last Form: {arcc.last_filing_type}")` **Output:** `Name: ARES CAPITAL CORP CIK: 1287750 File Number: 814-00663 State: MD Active: True Last Filing: 2024-11-05 Last Form: 10-Q` Portfolio Investments --------------------- BDCs disclose their portfolio holdings in the Schedule of Investments within their 10-K and 10-Q filings. ### Get Individual Investments Extract detailed investment positions from a BDC's latest filing: `arcc = bdcs.get_by_ticker("ARCC") investments = arcc.portfolio_investments() print(f"Total positions: {len(investments)}") print(f"Total fair value: ${investments.total_fair_value:,.0f}") print(f"Total cost: ${investments.total_cost:,.0f}")` **Output:** `Total positions: 1256 Total fair value: $26,800,000,000 Total cost: $25,100,000,000` ### Explore Investment Details Each investment includes detailed information: `# Get the largest investment inv = investments[0] print(f"Company: {inv.company_name}") print(f"Type: {inv.investment_type}") print(f"Fair Value: ${inv.fair_value:,.0f}") print(f"Cost: ${inv.cost:,.0f}") # For debt investments if inv.interest_rate: print(f"Interest Rate: {inv.interest_rate:.2%}") if inv.spread: print(f"Spread: {inv.spread:.2%}")` **Output:** `Company: Ivy Hill Asset Management, L.P. Type: First lien senior secured loan Fair Value: $1,915,300,000 Cost: $1,890,000,000 Interest Rate: 11.75% Spread: 5.75%` ### Filter Investments Find specific types of investments: `# First lien loans only first_lien = investments.filter(investment_type="First lien") print(f"First lien positions: {len(first_lien)}") # Search by company name software = investments.filter(company_name="software") print(f"Software companies: {len(software)}") # Large positions only from decimal import Decimal large = investments.filter(min_fair_value=Decimal('100000000')) print(f"Positions over $100M: {len(large)}")` ### Convert to DataFrame For analysis in pandas: `df = investments.to_dataframe() # Analyze by investment type print(df.groupby('investment_type')['fair_value'].sum().sort_values(ascending=False).head()) # Find largest positions print(df.nlargest(10, 'fair_value')[['company_name', 'investment_type', 'fair_value']])` ### Data Quality Check data completeness: `quality = investments.data_quality print(f"Fair value coverage: {quality.fair_value_coverage:.0%}") print(f"Cost coverage: {quality.cost_coverage:.0%}") print(f"Interest rate coverage: {quality.interest_rate_coverage:.0%}") print(f"Debt investments: {quality.debt_count}") print(f"Equity investments: {quality.equity_count}")` **Note:** Not all BDCs provide detailed XBRL data for individual investments. Use `has_detailed_investments()` to check: `if arcc.has_detailed_investments(): investments = arcc.portfolio_investments() else: print("This BDC only provides aggregate data")` Cross-BDC Analysis ------------------ Use SEC DERA bulk datasets for analysis across all BDCs. ### Fetch Quarterly Data `from edgar.bdc import fetch_bdc_dataset dataset = fetch_bdc_dataset(2024, 3) print(f"Period: {dataset.period}") print(f"BDCs: {dataset.num_companies}") print(f"SOI entries: {dataset.num_soi_entries:,}")` **Output:** `Period: 2024Q3 BDCs: 148 SOI entries: 106,715` ### Search for Portfolio Companies Find which BDCs hold a specific private company: `soi = dataset.schedule_of_investments results = soi.search("Ivy Hill") print(results)` **Output:** `company bdc_name bdc_cik fair_value 0 Ivy Hill Asset Management, L.P. ARES CAPITAL CORP 1287750 1915300000.0` ### Find Most Common Holdings Identify the most widely-held private companies: `top = soi.top_companies(10) print(top)` **Output:** `company num_bdcs total_fair_value bdc_names 0 OA Buyer, Inc. 6 322406000.0 BARINGS BDC, BARINGS CAPITAL... 1 MRI Software LLC 5 287500000.0 ARES CAPITAL, GOLUB CAPITAL...` ### Subset by BDC Get data for a specific BDC from the bulk dataset: `# By CIK arcc_soi = soi[1287750] print(f"ARCC entries: {len(arcc_soi)}") # By BDCEntity arcc = bdcs.get_by_ticker("ARCC") arcc_soi = soi[arcc]` ### Industry Analysis Analyze industry concentration: `summary = dataset.summary_by_industry() print(summary.head(10))` Integration with Company Objects -------------------------------- BDC entities connect to standard Company functionality: `arcc = bdcs.get_by_ticker("ARCC") # Get the Company object company = arcc.get_company() print(company) # Access filings filings = arcc.get_filings(form="10-K") latest_10k = filings[0] print(f"Latest 10-K: {latest_10k.filing_date}") # Get the Schedule of Investments statement soi_statement = arcc.schedule_of_investments() print(soi_statement)` Common Use Cases ---------------- ### Private Company Research Find all BDC exposure to a private company: `from edgar.bdc import fetch_bdc_dataset dataset = fetch_bdc_dataset(2024, 3) soi = dataset.schedule_of_investments # Search for the company results = soi.search("MRI Software") print(f"Found in {len(results)} BDC positions") print(f"Total exposure: ${results['fair_value'].sum():,.0f}") print(f"BDCs holding: {results['bdc_name'].unique().tolist()}")` ### BDC Portfolio Comparison Compare portfolio composition across BDCs: `from edgar.bdc import get_bdc_list bdcs = get_bdc_list() tickers = ["ARCC", "MAIN", "GBDC"] for ticker in tickers: bdc = bdcs.get_by_ticker(ticker) if bdc and bdc.has_detailed_investments(): inv = bdc.portfolio_investments() quality = inv.data_quality print(f"{ticker}: {len(inv)} positions, " f"{quality.debt_count} debt, {quality.equity_count} equity")` ### Yield Analysis Analyze interest rates across a BDC's debt portfolio: `arcc = bdcs.get_by_ticker("ARCC") investments = arcc.portfolio_investments() # Filter to debt with interest rates df = investments.to_dataframe() debt = df[df['interest_rate'].notna()] print(f"Average interest rate: {debt['interest_rate'].mean():.2%}") print(f"Rate range: {debt['interest_rate'].min():.2%} - {debt['interest_rate'].max():.2%}")` Data Sources ------------ The BDC module uses two SEC data sources: | Source | Content | Best For | | --- | --- | --- | | **SEC BDC Report** | List of all BDCs with file numbers | Finding and identifying BDCs | | **DERA Quarterly Datasets** | Pre-extracted Schedule of Investments | Cross-BDC analysis | | **Individual 10-K/10-Q** | Detailed XBRL investment data | Deep dive into single BDC | Performance Tips ---------------- 1. **Cache the BDC list**: `get_bdc_list()` is cached after first call 2. **Use bulk datasets for cross-BDC analysis**: Much faster than parsing individual filings 3. **Check data availability**: Use `has_detailed_investments()` before parsing 4. **Filter early**: Use the `filter()` method to reduce data before analysis Next Steps ---------- Now that you can access BDC data, learn how to: * **[Extract Financial Statements](https://edgartools.readthedocs.io/en/latest/guides/extract-statements/) ** - Get balance sheets and income statements * **[Query XBRL Facts](https://edgartools.readthedocs.io/en/latest/xbrl-querying/) ** - Deep dive into XBRL data * **[Company Facts API](https://edgartools.readthedocs.io/en/latest/guides/company-facts/) ** - Historical financial metrics Related Documentation --------------------- * **[Company Subsets](https://edgartools.readthedocs.io/en/latest/company-subsets/) ** - Create groups of companies for analysis * **[Finding Companies](https://edgartools.readthedocs.io/en/latest/guides/finding-companies/) ** - General company lookup Back to top --- # Entity API - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/entity-api-guide/#entitycompany-api-guide) Entity/Company API Guide ======================== This guide covers the Entity and Company API improvements in EdgarTools v5.3.0, including filer category identification and company icon retrieval. Table of Contents ----------------- 1. [Filer Category API](https://edgartools.readthedocs.io/en/stable/guides/entity-api-guide/#filer-category-api) 2. [Company Icon API](https://edgartools.readthedocs.io/en/stable/guides/entity-api-guide/#company-icon-api) 3. [Integration Examples](https://edgartools.readthedocs.io/en/stable/guides/entity-api-guide/#integration-examples) * * * Filer Category API ------------------ The SEC classifies public companies into filer categories based on their public float (market value of voting and non-voting common equity held by non-affiliates). EdgarTools v5.3.0 provides structured access to this classification data. ### Filer Status Classifications | Status | Public Float Threshold | Filing Deadlines | | --- | --- | --- | | Large Accelerated Filer | \>= $700 million | 60 days (10-K), 40 days (10-Q) | | Accelerated Filer | \>= $75M and < $700M | 75 days (10-K), 40 days (10-Q) | | Non-Accelerated Filer | < $75 million | 90 days (10-K), 45 days (10-Q) | ### Filer Qualifications In addition to the base status, companies may have these qualifications: * **Smaller Reporting Company (SRC)**: < $250M public float OR < $100M annual revenue * **Emerging Growth Company (EGC)**: < $1.235B revenue, IPO within 5 years ### Quick Usage `from edgar import Company company = Company("AAPL") # Boolean property checks if company.is_large_accelerated_filer: print("Large accelerated filer - earliest filing deadlines") if company.is_smaller_reporting_company: print("Qualifies for scaled disclosure requirements") if company.is_emerging_growth_company: print("May use EGC accommodations")` ### Available Properties | Property | Type | Description | | --- | --- | --- | | `filer_category` | `FilerCategory` | Full parsed category object | | `is_large_accelerated_filer` | `bool` | Public float >= $700M | | `is_accelerated_filer` | `bool` | Public float >= $75M and < $700M | | `is_non_accelerated_filer` | `bool` | Public float < $75M | | `is_smaller_reporting_company` | `bool` | Qualifies as SRC | | `is_emerging_growth_company` | `bool` | Qualifies as EGC | ### Working with FilerCategory Object For more detailed analysis, use the `filer_category` property: `from edgar import Company from edgar.enums import FilerStatus, FilerQualification company = Company("AAPL") category = company.filer_category # Access the base status enum status = category.status # FilerStatus.LARGE_ACCELERATED # Check specific status if category.status == FilerStatus.LARGE_ACCELERATED: print("Large accelerated filer") # Get all qualifications as a list qualifications = category.qualifications # Returns: [FilerQualification.SMALLER_REPORTING_COMPANY, ...] # String representation (original SEC format) print(str(category)) # "Large accelerated filer"` ### Parsing SEC Category Strings The SEC returns category as a compound string with `|` separator: `from edgar.enums import FilerCategory # Parse SEC format strings directly category = FilerCategory.from_string("Accelerated filer | Smaller reporting company") print(category.status) # FilerStatus.ACCELERATED print(category.is_smaller_reporting_company) # True print(category.is_emerging_growth_company) # False # Handle compound qualifications category = FilerCategory.from_string( "Non-accelerated filer | Smaller reporting company | Emerging growth company" ) print(len(category.qualifications)) # 2` ### Enums Reference `from edgar.enums import FilerStatus, FilerQualification # FilerStatus values FilerStatus.LARGE_ACCELERATED # "Large accelerated filer" FilerStatus.ACCELERATED # "Accelerated filer" FilerStatus.NON_ACCELERATED # "Non-accelerated filer" # FilerQualification values FilerQualification.SMALLER_REPORTING_COMPANY # "Smaller reporting company" FilerQualification.EMERGING_GROWTH_COMPANY # "Emerging growth company"` See it live on edgar.tools The code above checks filer categories programmatically. **edgar.tools** shows the same company metadata visually — filer status, industry classification, shares outstanding, and public float for any SEC entity. * **[See Apple's company profile →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=entity-api) ** * **[Search 940K+ SEC entities →](https://app.edgar.tools/companies?utm_source=edgartools-docs&utm_medium=see-live&utm_content=entity-api) ** Free tier available. Also includes a REST API for programmatic entity lookups. [API docs →](https://app.edgar.tools/docs?utm_source=edgartools-docs&utm_medium=see-live&utm_content=entity-api) * * * Company Icon API ---------------- EdgarTools provides access to company logo/icon images via the `get_icon_from_ticker` function. Icons are sourced from the [nvstly/icons](https://github.com/nvstly/icons) repository on GitHub. ### Basic Usage `from edgar import get_icon_from_ticker # Get icon as PNG bytes icon_bytes = get_icon_from_ticker("AAPL") if icon_bytes: # Save to file with open("apple_logo.png", "wb") as f: f.write(icon_bytes)` ### Function Signature `def get_icon_from_ticker(ticker: str) -> Optional[bytes]: """ Download an icon for a given ticker as a PNG image, if available. Args: ticker: Stock ticker symbol (e.g., "AAPL", "MSFT", "BRK-B") Returns: bytes: PNG image data if icon exists None: If no icon is available for this ticker Raises: ValueError: If ticker is invalid (empty, contains invalid characters) """` ### Handling Hyphenated Tickers As of v5.3.0, hyphenated tickers are fully supported: `# Berkshire Hathaway Class B shares icon = get_icon_from_ticker("BRK-B") # Works correctly # The function strips hyphens internally since the icon repository # stores icons as BRKB.png, not BRK-B.png` ### Validation Rules The ticker must: - Be a non-empty string - Contain only alphabetic characters (A-Z) and hyphens (-) - Not contain numbers, spaces, or special characters `# Valid tickers get_icon_from_ticker("AAPL") # OK get_icon_from_ticker("BRK-B") # OK (hyphenated) get_icon_from_ticker("msft") # OK (case insensitive) # Invalid tickers - raise ValueError get_icon_from_ticker("") # Empty string get_icon_from_ticker("AAPL123") # Contains numbers get_icon_from_ticker("AA PL") # Contains space get_icon_from_ticker(None) # Not a string` ### Caching The function uses LRU caching (maxsize=4) to avoid repeated network requests: `# First call fetches from network icon1 = get_icon_from_ticker("AAPL") # Subsequent calls return cached result icon2 = get_icon_from_ticker("AAPL") # Instant, no network call` ### Building Icon URLs If you need the URL directly (e.g., for client-side rendering): `from edgar.reference.tickers import get_ticker_icon_url url = get_ticker_icon_url("AAPL") # Returns: "https://raw.githubusercontent.com/nvstly/icons/main/ticker_icons/AAPL.png"` **Note**: For hyphenated tickers, you need to strip the hyphen manually for the URL: `ticker = "BRK-B" url = f"https://raw.githubusercontent.com/nvstly/icons/main/ticker_icons/{ticker.replace('-', '').upper()}.png" # Returns: "https://raw.githubusercontent.com/nvstly/icons/main/ticker_icons/BRKB.png"` * * * Integration Examples -------------------- ### SaaS Dashboard: Company Card Component `from edgar import Company, get_icon_from_ticker import base64 def get_company_card_data(ticker: str) -> dict: """ Build company card data for a SaaS dashboard. """ company = Company(ticker) # Get icon as base64 for embedding in HTML/JSON icon_bytes = get_icon_from_ticker(ticker) icon_base64 = base64.b64encode(icon_bytes).decode() if icon_bytes else None # Determine regulatory tier for UI badges if company.is_large_accelerated_filer: regulatory_tier = "Large Cap" tier_color = "blue" elif company.is_accelerated_filer: regulatory_tier = "Mid Cap" tier_color = "green" else: regulatory_tier = "Small Cap" tier_color = "gray" # Build badges list badges = [regulatory_tier] if company.is_smaller_reporting_company: badges.append("SRC") if company.is_emerging_growth_company: badges.append("EGC") return { "ticker": ticker, "name": company.name, "cik": company.cik, "icon_base64": icon_base64, "icon_url": f"data:image/png;base64,{icon_base64}" if icon_base64 else None, "regulatory_tier": regulatory_tier, "tier_color": tier_color, "badges": badges, "filer_category_raw": str(company.filer_category), } # Usage card = get_company_card_data("AAPL") # { # "ticker": "AAPL", # "name": "Apple Inc.", # "cik": 320193, # "icon_base64": "iVBORw0KGgo...", # "regulatory_tier": "Large Cap", # "tier_color": "blue", # "badges": ["Large Cap"], # "filer_category_raw": "Large accelerated filer" # }` ### Filtering Companies by Filer Status `from edgar import Company def filter_by_filer_status(tickers: list[str], status: str) -> list[str]: """ Filter tickers by their SEC filer status. Args: tickers: List of ticker symbols status: One of "large_accelerated", "accelerated", "non_accelerated" Returns: List of tickers matching the specified status """ results = [] for ticker in tickers: try: company = Company(ticker) match status: case "large_accelerated": if company.is_large_accelerated_filer: results.append(ticker) case "accelerated": if company.is_accelerated_filer: results.append(ticker) case "non_accelerated": if company.is_non_accelerated_filer: results.append(ticker) except Exception: continue # Skip invalid tickers return results # Find all emerging growth companies def find_egc_companies(tickers: list[str]) -> list[str]: return [t for t in tickers if Company(t).is_emerging_growth_company]` ### API Response Builder `from edgar import Company, get_icon_from_ticker from edgar.enums import FilerStatus import json def build_company_api_response(ticker: str) -> dict: """ Build a complete API response for company data. """ company = Company(ticker) category = company.filer_category return { "company": { "ticker": ticker, "name": company.name, "cik": company.cik, }, "filer_classification": { "status": category.status.value if category.status else None, "status_code": category.status.name if category.status else None, "is_large_accelerated": company.is_large_accelerated_filer, "is_accelerated": company.is_accelerated_filer, "is_non_accelerated": company.is_non_accelerated_filer, }, "qualifications": { "smaller_reporting_company": company.is_smaller_reporting_company, "emerging_growth_company": company.is_emerging_growth_company, }, "branding": { "icon_available": get_icon_from_ticker(ticker) is not None, "icon_url": f"/api/company/{ticker}/icon", # Your API endpoint }, "raw_sec_category": str(category), } # Example output for AAPL: # { # "company": {"ticker": "AAPL", "name": "Apple Inc.", "cik": 320193}, # "filer_classification": { # "status": "Large accelerated filer", # "status_code": "LARGE_ACCELERATED", # "is_large_accelerated": True, # "is_accelerated": False, # "is_non_accelerated": False # }, # "qualifications": { # "smaller_reporting_company": False, # "emerging_growth_company": False # }, # "branding": { # "icon_available": True, # "icon_url": "/api/company/AAPL/icon" # }, # "raw_sec_category": "Large accelerated filer" # }` ### Flask/FastAPI Icon Endpoint `# Flask example from flask import Flask, Response, abort from edgar import get_icon_from_ticker app = Flask(__name__) @app.route("/api/company//icon") def company_icon(ticker: str): try: icon_bytes = get_icon_from_ticker(ticker.upper()) if icon_bytes is None: abort(404, description="Icon not available for this ticker") return Response(icon_bytes, mimetype="image/png") except ValueError as e: abort(400, description=str(e))` `# FastAPI example from fastapi import FastAPI, HTTPException from fastapi.responses import Response from edgar import get_icon_from_ticker app = FastAPI() @app.get("/api/company/{ticker}/icon") async def company_icon(ticker: str): try: icon_bytes = get_icon_from_ticker(ticker.upper()) if icon_bytes is None: raise HTTPException(status_code=404, detail="Icon not available") return Response(content=icon_bytes, media_type="image/png") except ValueError as e: raise HTTPException(status_code=400, detail=str(e))` * * * Notes and Limitations --------------------- ### Filer Category API * Filer category data comes from SEC submission metadata * The `filer_category` property is cached per Company instance * Some older or unusual entities may not have category data (returns empty `FilerCategory`) ### Icon API * Icons are sourced from a third-party GitHub repository (nvstly/icons) * Not all tickers have icons available - check for `None` return * The repository focuses on popular US stocks * Icon format is PNG * Results are cached (LRU cache, maxsize=4) * Network errors (other than 404) are propagated as exceptions ### Performance Considerations `# For batch operations, consider caching at the application level from functools import lru_cache @lru_cache(maxsize=1000) def get_company_data_cached(ticker: str): company = Company(ticker) return { "name": company.name, "is_large_accelerated": company.is_large_accelerated_filer, # ... etc }` Back to top --- # Search & Filter - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/searching-filings/#search-and-filter-sec-filings-by-form-date-and-company) Search and Filter SEC Filings by Form, Date, and Company ======================================================== Learn how to find the exact SEC filings you need using various search criteria and filtering methods. Prerequisites ------------- * Understanding of SEC filing types (10-K, 10-Q, 8-K, etc.) Basic Filing Search ------------------- ### Get Recent Filings Start with the most recent filings across all companies: `from edgar import get_filings # Get the 50 most recent filings recent_filings = get_filings() # Display basic information for filing in recent_filings[:5]: print(f"{filing.form}: {filing.company_name} ({filing.filing_date})")` **Output:** `10-Q: Apple Inc. (2024-05-02) 8-K: Microsoft Corporation (2024-05-01) 10-K: Amazon.com Inc (2024-04-30) 13F-HR: Berkshire Hathaway Inc (2024-04-29) 4: Tesla Inc (2024-04-28)` ### Search by Filing Type Find specific types of SEC forms: `# Get recent 10-K annual reports annual_reports = get_filings(form="10-K").head(20) # Get multiple form types quarterly_and_annual = get_filings(form=["10-K", "10-Q"]) # Exclude amendments (filings ending in /A) original_filings = get_filings(form="10-K", amendments=False).head(20) print(f"Found {len(annual_reports)} annual reports")` Search by Date Range -------------------- ### Specific Date `# Get all filings from a specific date filings_jan_1 = get_filings(filing_date="2024-01-01") print(f"Found {len(filings_jan_1)} filings on 2024-01-01")` ### Date Ranges `# Get filings from a date range q1_filings = get_filings(filing_date="2024-01-01:2024-03-31") # Get filings after a specific date recent_filings = get_filings(filing_date="2024-01-01:") # Get filings before a specific date older_filings = get_filings(filing_date=":2023-12-31") print(f"Q1 2024 filings: {len(q1_filings)}")` ### Year and Quarter Search Calendar Year vs Fiscal Year The `year` and `quarter` parameters refer to **when the filing was submitted to the SEC** (calendar year), **not** the fiscal year the filing covers. For example, a company with a fiscal year ending March 31, 2024 would file their annual 10-K in May or June 2024. Using `get_filings(2024)` would find this filing because it was **filed** in calendar year 2024, even though the 10-K covers fiscal year 2024. To find filings by fiscal year, use the company's `get_filings()` method and filter by the filing's fiscal period information available in the XBRL data. `# Get filings for entire calendar year (by filing date) filings_2023 = get_filings(2023) # Get filings for specific calendar quarter q4_2023 = get_filings(2023, 4) # Get multiple quarters q3_q4_2023 = get_filings(2023, [3, 4]) # Get multiple years multi_year = get_filings([2022, 2023]) # Get range of years (excludes end year) decade_filings = get_filings(range(2010, 2021)) print(f"2023 filings: {len(filings_2023)}") print(f"Q4 2023 filings: {len(q4_2023)}")` Company-Specific Filing Search ------------------------------ ### Get All Company Filings `from edgar import Company # Get all filings for a company apple = Company("AAPL") all_apple_filings = apple.get_filings() print(f"Apple has {len(all_apple_filings)} total filings")` ### Filter Company Filings `# Get specific form types for a company apple_10k = apple.get_filings(form="10-K") apple_quarterly = apple.get_filings(form=["10-Q", "10-K"]) # Get XBRL filings only apple_xbrl = apple.get_filings(is_xbrl=True) # Get inline XBRL filings apple_ixbrl = apple.get_filings(is_inline_xbrl=True) print(f"Apple 10-K filings: {len(apple_10k)}") print(f"Apple XBRL filings: {len(apple_xbrl)}")` ### Get Latest Filing `# Get the most recent filing of a specific type latest_10k = apple.get_filings(form="10-K").latest() latest_10q = apple.get_filings(form="10-Q").latest() print(f"Latest 10-K: {latest_10k.filing_date}") print(f"Latest 10-Q: {latest_10q.filing_date}") # Chain the calls for conciseness latest_annual = Company("MSFT").get_filings(form="10-K").latest()` See it live on edgar.tools The code above searches filings by form type, date, and company. **edgar.tools** puts the same search in a visual interface — filter by form, date range, and company with results updating in real time. * **[Browse the real-time filing stream →](https://app.edgar.tools/filings?utm_source=edgartools-docs&utm_medium=see-live&utm_content=searching-filings) ** * **[Search Apple's full filing history →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=searching-filings) ** Also includes a REST API with filing search endpoints. Free tier: 100 API calls/day. [Pricing →](https://app.edgar.tools/pricing?utm_source=edgartools-docs&utm_medium=see-live&utm_content=searching-filings) Advanced Filtering ------------------ ### Filter by Multiple Criteria `# Get Apple's 10-K filings from 2023 that are XBRL apple_filtered = apple.get_filings( form="10-K", is_xbrl=True ).filter(filing_date="2023-01-01:2023-12-31") print(f"Filtered results: {len(apple_filtered)}")` ### Filter by Accession Number `# Find specific filing by accession number specific_filing = apple.get_filings( accession_number="0000320193-23-000106" ) print(f"Found filing: {specific_filing[0].form}")` ### Filter by File Number `# Filter by SEC file number file_filtered = apple.get_filings( file_number="001-36743" ) print(f"Filings with file number: {len(file_filtered)}")` Cross-Company Search -------------------- ### Search by Industry `# Get recent filings and filter by company characteristics all_filings = get_filings() # Filter for technology companies (requires loading each company) tech_filings = [] for filing in all_filings[:100]: # Limit for performance try: company = Company(filing.cik) if "software" in company.industry.lower() or "computer" in company.industry.lower(): tech_filings.append(filing) except: continue print(f"Found {len(tech_filings)} filings from tech companies")` ### Search by Exchange `# Filter existing filings by exchange nasdaq_filings = all_filings.filter(exchange="NASDAQ") nyse_filings = all_filings.filter(exchange="NYSE") print(f"NASDAQ filings: {len(nasdaq_filings)}") print(f"NYSE filings: {len(nyse_filings)}")` ### Search by Ticker List `# Get filings for multiple specific companies tickers = ["AAPL", "MSFT", "GOOGL", "AMZN"] ticker_filings = all_filings.filter(ticker=tickers) print(f"Filings from specified tickers: {len(ticker_filings)}")` Specialized Filing Searches --------------------------- ### Insider Trading Filings `# Get recent insider trading filings insider_filings = get_filings(form=["3", "4", "5"]) print("Recent insider filings:") for filing in insider_filings[:10]: print(f" Form {filing.form}: {filing.company_name} ({filing.filing_date})")` ### Fund Holdings (13F) `# Get recent 13F filings (institutional investment managers) fund_filings = get_filings(form="13F-HR") print("Recent fund holdings filings:") for filing in fund_filings: print(f" {filing.company_name}: {filing.filing_date}")` ### Material Events (8-K) `# Get recent 8-K filings (material corporate events) event_filings = get_filings(form="8-K") print("Recent material events:") for filing in event_filings[:10]: print(f" {filing.company_name}: {filing.filing_date}")` ### IPO and Registration Statements `# Get S-1 filings (IPO registrations) ipo_filings = get_filings(form="S-1") print("Recent IPO filings:") for filing in ipo_filings: print(f" {filing.company_name}: {filing.filing_date}")` Working with Search Results --------------------------- ### Subset and Sample `filings = get_filings(form="10-K") # Get first 10 results first_ten = filings.head(10) # Get last 10 results last_ten = filings.tail(10) # Get random sample of 5 results random_sample = filings.sample(5) print(f"Total: {len(filings)}, Sample: {len(random_sample)}")` ### Convert to Pandas DataFrame `import pandas as pd # Convert filings to DataFrame for analysis filings_df = filings.to_pandas() # Analyze filing patterns filing_counts = filings_df.groupby(['form', 'company_name']).size() print("Filing counts by company and form:") print(filing_counts.head(10))` ### Access Underlying Data `# Access the PyArrow table directly import pyarrow as pa filings = get_filings(form="10-K") data_table: pa.Table = filings.data # Convert to Pandas for advanced analysis df = data_table.to_pandas() print(f"Columns available: {df.columns.tolist()}")` Performance Optimization ------------------------ ### Efficient Searching `# More efficient: Use specific parameters in get_filings() efficient = get_filings(form="10-K", filing_date="2023-01-01:") # Less efficient: Get all then filter inefficient = get_filings().filter(form="10-K").filter(filing_date="2023-01-01:") print(f"Efficient approach found: {len(efficient)} filings")` ### Caching Results `# Store frequently used searches apple = Company("AAPL") apple_10k_cache = apple.get_filings(form="10-K") # Reuse cached results for different analyses recent_10k = apple_10k_cache.head(5) oldest_10k = apple_10k_cache.tail(5)` Error Handling -------------- ### Handle Missing Data `try: filings = get_filings(form="INVALID-FORM") print(f"Found {len(filings)} filings") except Exception as e: print(f"Error searching filings: {e}")` ### Validate Search Results `filings = get_filings(form="10-K", limit=10) if len(filings) == 0: print("No filings found matching criteria") else: print(f"Found {len(filings)} filings") # Verify first result first_filing = filings[0] print(f"First result: {first_filing.form} from {first_filing.company_name}")` Common Search Patterns ---------------------- ### Earnings Season Analysis `# Find quarterly reports filed in typical earnings periods earnings_dates = [ "2024-01-15:2024-02-15", # Q4 earnings "2024-04-15:2024-05-15", # Q1 earnings "2024-07-15:2024-08-15", # Q2 earnings "2024-10-15:2024-11-15" # Q3 earnings ] earnings_filings = [] for date_range in earnings_dates: filings = get_filings(form="10-Q", filing_date=date_range) earnings_filings.extend(filings) print(f"Found {len(earnings_filings)} earnings period filings")` ### M&A Activity Monitoring `# Look for 8-K filings that might indicate M&A activity ma_filings = get_filings(form="8-K") # Filter for potential M&A keywords (requires examining filing content) potential_ma = [] for filing in ma_filings[:50]: # Limit for performance try: text = filing.text() if any(keyword in text.lower() for keyword in ['acquisition', 'merger', 'tender offer', 'purchase agreement']): potential_ma.append(filing) except: continue print(f"Found {len(potential_ma)} potential M&A filings")` Next Steps ---------- Now that you can search for filings effectively, learn how to: * **[Filter Filings by Date/Type](https://edgartools.readthedocs.io/en/latest/guides/filtering-filings/) ** - Advanced filtering techniques * **[Access Filing Attachments](https://edgartools.readthedocs.io/en/latest/guides/filing-attachments/) ** - Get supporting documents Related Documentation --------------------- * **[Filing API Reference](https://edgartools.readthedocs.io/en/latest/api/filing/) ** - Complete Filing class documentation * **[Filings API Reference](https://edgartools.readthedocs.io/en/latest/api/filings/) ** - Filings collection methods * **[Working with Filings](https://edgartools.readthedocs.io/en/latest/guides/working-with-filing/) ** - Original filing documentation Back to top --- # Understanding 10-K, 10-Q, and 8-K Report Objects in Python - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/concepts/data-objects/#understanding-10-k-10-q-and-8-k-report-objects-in-python) Understanding 10-K, 10-Q, and 8-K Report Objects in Python ========================================================== Introduction ------------ One of the most powerful features of edgartools is its Data Objects system. This system transforms raw SEC filing data into structured, easy-to-use Python objects that expose filing-specific properties and methods. Instead of dealing with complex HTML, XML, or XBRL parsing yourself, Data Objects handle all the heavy lifting, allowing you to focus on analysis rather than data extraction. This guide explains the conceptual framework behind Data Objects, how they work under the hood, and how to leverage them effectively in your SEC data analysis workflows. The Problem Data Objects Solve ------------------------------ SEC filings are notoriously complex documents: * They contain a mix of structured and unstructured data * They use different formats (HTML, XML, XBRL) depending on filing type and date * Their structure evolves over time as SEC requirements change * They often contain inconsistencies in formatting and organization * They require domain knowledge to interpret correctly Without Data Objects, working with SEC filings would require: 1. Downloading raw filing documents 2. Writing custom parsers for each filing type 3. Handling edge cases and inconsistencies 4. Extracting and organizing the data manually 5. Converting data into usable formats for analysis Data Objects eliminate these challenges by providing a consistent, intuitive interface to SEC filing data, regardless of the underlying format or structure. The Data Objects Architecture ----------------------------- ### Core Principles The Data Objects system is built on several key principles: 1. **Type-Specific Interfaces**: Each filing type has its own specialized interface that exposes only the relevant properties and methods. 2. **Lazy Parsing**: Content is parsed on-demand to minimize memory usage and processing time. 3. **Consistent Access Patterns**: Similar data is accessed through consistent patterns across different filing types. 4. **Rich Metadata**: Each object includes metadata about the filing, such as dates, filer information, and document structure. 5. **Transformation Capabilities**: Data can be easily transformed into formats like pandas DataFrames for analysis. ### Object Hierarchy Data Objects follow a hierarchical structure: `Filing (base class) ├── CompanyFiling │ ├── TenK (10-K Annual Report) │ ├── TenQ (10-Q Quarterly Report) │ └── EightK (8-K Current Report) ├── OwnershipFiling │ ├── Form3 (Initial Ownership) │ ├── Form4 (Changes in Ownership) │ └── Form5 (Annual Ownership Summary) ├── InvestmentFiling │ └── ThirteenF (13F Holdings Report) └── Other specialized filing types` Each object in this hierarchy inherits common functionality while adding specialized features for its filing type. How Data Objects Work --------------------- ### The Creation Process When you call the `.obj()` method on a Filing object, the following process occurs: 1. **Filing Type Detection**: The system identifies the filing type based on the form type and content. 2. **Parser Selection**: The appropriate parser is selected for that filing type. 3. **Object Instantiation**: A new Data Object of the correct type is created. 4. **Initial Parsing**: Basic metadata is parsed immediately. 5. **Lazy Loading Setup**: More complex content is set up for on-demand parsing. ### Parsing Strategies Data Objects use different parsing strategies depending on the filing type: * **HTML Parsing**: For narrative sections like business descriptions and risk factors * **XML Parsing**: For structured data like ownership transactions and fund holdings * **XBRL Processing**: For financial statements and other tagged financial data * **Table Extraction**: For tabular data embedded in filings * **Text Processing**: For extracting plain text from complex HTML structures These strategies are applied automatically based on the content being accessed. Working with Data Objects ------------------------- ### Common Patterns Across all Data Objects, you'll find these common patterns: 1. **Property Access**: Access filing sections or data through properties (e.g., `tenk.risk_factors`, `tenk.auditor`, `tenk.subsidiaries`, `tenk.reports`) 2. **Method Calls**: Perform operations on the data (e.g., `form4.get_net_shares_traded()`) 3. **Dictionary-Like Access**: Access specific items by key (e.g., `eightk["Item 2.01"]`) 4. **Iteration**: Iterate over collections within the filing (e.g., `for holding in thirteen_f.infotable`) 5. **Conversion**: Transform data into other formats (e.g., `balance_sheet.to_dataframe()`) ### Object Persistence Data Objects are designed to be lightweight and don't persist the entire filing content in memory. Instead, they: 1. Store references to the original filing content 2. Parse specific sections only when accessed 3. Cache parsed results to avoid repeated parsing 4. Release memory when no longer needed This approach allows you to work with very large filings efficiently. Advanced Usage Patterns ----------------------- ### Combining Multiple Data Objects You can combine data from multiple Data Objects for more sophisticated analysis: `# Compare financial data across quarters company = Company("AAPL") filings = company.get_filings(form=["10-K", "10-Q"]).head(5) data_objects = [filing.obj() for filing in filings] # Extract revenue from each filing revenues = [] for obj in data_objects: if hasattr(obj, "income_statement"): period_end = obj.period_end_date revenue = obj.income_statement.get_value("Revenues") revenues.append((period_end, revenue)) # Sort by date and analyze trend revenues.sort(key=lambda x: x[0])` See it live on edgar.tools The code above combines multiple Data Objects for cross-period analysis. **edgar.tools** does this automatically — multi-year financials, disclosure timelines, and filing comparisons for any company, with export to Excel or PDF. * **[See Apple's financials across multiple years →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=data-objects-concepts) ** * **[Browse 12 XBRL disclosure topics →](https://app.edgar.tools/disclosures?utm_source=edgartools-docs&utm_medium=see-live&utm_content=data-objects-concepts) ** Free tier available. Also includes a REST API and hosted MCP server for AI integrations. ### Custom Data Extraction You can extend Data Objects with your own extraction logic: `def extract_cybersecurity_risks(tenk): """Extract cybersecurity-related content from risk factors.""" if not hasattr(tenk, "risk_factors"): return None risk_text = tenk.risk_factors cyber_keywords = ["cyber", "hack", "breach", "data security", "privacy"] # Find paragraphs containing cyber keywords paragraphs = risk_text.split("\n\n") cyber_paragraphs = [p for p in paragraphs if any(k in p.lower() for k in cyber_keywords)] return cyber_paragraphs # Apply to a 10-K tenk = company.latest("10-K").obj() cyber_risks = extract_cybersecurity_risks(tenk)` ### Batch Processing For processing many filings efficiently: `# Process all 8-Ks from the past year company = Company("MSFT") filings = company.get_filings(form="8-K", start_date="2024-01-01") # Extract all press releases all_press_releases = [] for filing in filings: try: eightk = filing.obj() if eightk.has_press_release: for pr in eightk.press_releases: all_press_releases.append({ "date": eightk.date_of_report, "title": pr.title, "content": pr.content }) except Exception as e: print(f"Error processing filing {filing.accession_number}: {e}") print(f"Found {len(all_press_releases)} press releases")` Common Challenges and Solutions ------------------------------- ### Challenge: Handling Missing Data Not all filings contain all expected sections or data points: `# Safe access pattern tenk = filing.obj() if hasattr(tenk, "risk_factors") and tenk.risk_factors: # Process risk factors pass else: print("No risk factors section found") # For financial data try: revenue = income_stmt.get_value("Revenues") except ValueError: revenue = income_stmt.get_value("RevenueFromContractWithCustomerExcludingAssessedTax") except: revenue = None` ### Challenge: Handling Format Changes SEC filing formats evolve over time: `# Version-aware code tenk = filing.obj() filing_year = tenk.period_end_date.year if filing_year >= 2021: # Use newer XBRL taxonomy concepts revenue = income_stmt.get_value("RevenueFromContractWithCustomerExcludingAssessedTax") else: # Use older concepts revenue = income_stmt.get_value("Revenues")` ### Challenge: Processing Large Filings Some filings (especially 10-Ks) can be very large: `# Memory-efficient processing tenk = filing.obj() # Process one section at a time sections = ["business", "risk_factors", "management_discussion"] for section_name in sections: if hasattr(tenk, section_name): section = getattr(tenk, section_name) # Process section # ... # Explicitly delete to free memory del section` Best Practices -------------- ### 1\. Use the Right Object for the Task Choose the most specific Data Object for your needs: * Use `TenK`/`TenQ` for financial statement analysis * Use `TenK` for auditor info (`tenk.auditor`), subsidiaries (`tenk.subsidiaries`), and XBRL report pages (`tenk.reports`) * Use `EightK` for event monitoring * Use `Form4` for insider trading analysis * Use `ThirteenF` for fund holdings analysis ### 2\. Leverage Built-in Methods Data Objects include many helpful methods that save you from writing custom code: `# Instead of parsing manually: form4 = filing.obj() net_shares = form4.get_net_shares_traded() # Built-in method # Instead of calculating manually: thirteen_f = filing.obj() top_10 = thirteen_f.get_top_holdings(10) # Built-in method` ### 3\. Handle Errors Gracefully SEC filings can have inconsistencies that cause parsing errors: `try: data_obj = filing.obj() # Work with the object except Exception as e: print(f"Error parsing filing {filing.accession_number}: {e}") # Fall back to simpler access methods text = filing.text` ### 4\. Use Local Storage * Data Objects parse filing content on-demand * Large filings (like 10-Ks) may take a few seconds to parse * Consider using local storage for batch processing Conclusion ---------- Data Objects are the heart of edgartools' power and usability. By abstracting away the complexities of SEC filing formats and structures, they allow you to focus on analysis rather than data extraction. Understanding how Data Objects work and how to use them effectively will help you build more powerful, efficient, and maintainable SEC data analysis workflows. Whether you're analyzing financial statements, tracking insider trading, or researching investment funds, Data Objects provide a consistent, intuitive interface that makes working with SEC data a breeze. Additional Resources -------------------- * [Working with Financial Statements](https://edgartools.readthedocs.io/en/latest/guides/extract-statements/) * [Current Events (8-K)](https://edgartools.readthedocs.io/en/latest/eightk-filings/) * [Analyzing Insider Trading](https://edgartools.readthedocs.io/en/latest/guides/track-form4/) * [Institutional Holdings (13F)](https://edgartools.readthedocs.io/en/latest/guides/thirteenf-data-object-guide/) Back to top --- # Business Development Companies - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/bdc-guide/#business-development-companies) Business Development Companies ============================== Access and analyze Business Development Companies (BDCs) - specialized investment companies that invest in small and mid-sized private companies. What Are BDCs? -------------- Business Development Companies are closed-end investment funds that: * Invest in small and mid-sized private U.S. companies * Provide financing (loans) and equity investments * Must distribute 90%+ of taxable income as dividends * File with the SEC under file numbers starting with "814-" BDCs provide a unique window into private credit markets through their quarterly Schedule of Investments disclosures. Finding BDCs ------------ ### List All BDCs Get the complete list of ~196 BDCs from the SEC BDC Report: `from edgar.bdc import get_bdc_list bdcs = get_bdc_list() print(f"Found {len(bdcs)} BDCs")` **Output:** `Found 196 BDCs` ### Get a Specific BDC Find a BDC by ticker or CIK: `from edgar.bdc import get_bdc_list bdcs = get_bdc_list() # By ticker arcc = bdcs.get_by_ticker("ARCC") print(arcc.name) # ARES CAPITAL CORP # By CIK main = bdcs.get_by_cik(1396440) print(main.name) # MAIN STREET CAPITAL CORP` ### Search for BDCs Use fuzzy search to find BDCs by name or ticker: `from edgar.bdc import find_bdc # Search by name results = find_bdc("Ares") print(results[0].name) # ARES CAPITAL CORP # Search by ticker results = find_bdc("MAIN") print(results[0].name) # MAIN STREET CAPITAL CORP` **Key points:** - Search is case-insensitive - Supports partial name matching - Returns ranked results by relevance ### Filter BDCs Filter by state or activity status: `bdcs = get_bdc_list() # Active BDCs only (filed within last 18 months) active_bdcs = bdcs.filter(active=True) print(f"Active BDCs: {len(active_bdcs)}") # BDCs in New York ny_bdcs = bdcs.filter(state='NY') print(f"NY-based BDCs: {len(ny_bdcs)}") # Combine filters ny_active = bdcs.filter(state='NY', active=True)` ### Check if a Company is a BDC Verify whether a CIK belongs to a BDC: `from edgar.bdc import is_bdc_cik is_bdc_cik(1287750) # True - ARCC is a BDC is_bdc_cik(320193) # False - Apple is not a BDC` BDC Properties -------------- Each BDC entity provides useful information: `arcc = bdcs.get_by_ticker("ARCC") # Basic info print(f"Name: {arcc.name}") print(f"CIK: {arcc.cik}") print(f"File Number: {arcc.file_number}") # 814-00663 print(f"State: {arcc.state}") print(f"Active: {arcc.is_active}") # Filing info print(f"Last Filing: {arcc.last_filing_date}") print(f"Last Form: {arcc.last_filing_type}")` **Output:** `Name: ARES CAPITAL CORP CIK: 1287750 File Number: 814-00663 State: MD Active: True Last Filing: 2024-11-05 Last Form: 10-Q` Portfolio Investments --------------------- BDCs disclose their portfolio holdings in the Schedule of Investments within their 10-K and 10-Q filings. ### Get Individual Investments Extract detailed investment positions from a BDC's latest filing: `arcc = bdcs.get_by_ticker("ARCC") investments = arcc.portfolio_investments() print(f"Total positions: {len(investments)}") print(f"Total fair value: ${investments.total_fair_value:,.0f}") print(f"Total cost: ${investments.total_cost:,.0f}")` **Output:** `Total positions: 1256 Total fair value: $26,800,000,000 Total cost: $25,100,000,000` ### Explore Investment Details Each investment includes detailed information: `# Get the largest investment inv = investments[0] print(f"Company: {inv.company_name}") print(f"Type: {inv.investment_type}") print(f"Fair Value: ${inv.fair_value:,.0f}") print(f"Cost: ${inv.cost:,.0f}") # For debt investments if inv.interest_rate: print(f"Interest Rate: {inv.interest_rate:.2%}") if inv.spread: print(f"Spread: {inv.spread:.2%}")` **Output:** `Company: Ivy Hill Asset Management, L.P. Type: First lien senior secured loan Fair Value: $1,915,300,000 Cost: $1,890,000,000 Interest Rate: 11.75% Spread: 5.75%` ### Filter Investments Find specific types of investments: `# First lien loans only first_lien = investments.filter(investment_type="First lien") print(f"First lien positions: {len(first_lien)}") # Search by company name software = investments.filter(company_name="software") print(f"Software companies: {len(software)}") # Large positions only from decimal import Decimal large = investments.filter(min_fair_value=Decimal('100000000')) print(f"Positions over $100M: {len(large)}")` ### Convert to DataFrame For analysis in pandas: `df = investments.to_dataframe() # Analyze by investment type print(df.groupby('investment_type')['fair_value'].sum().sort_values(ascending=False).head()) # Find largest positions print(df.nlargest(10, 'fair_value')[['company_name', 'investment_type', 'fair_value']])` ### Data Quality Check data completeness: `quality = investments.data_quality print(f"Fair value coverage: {quality.fair_value_coverage:.0%}") print(f"Cost coverage: {quality.cost_coverage:.0%}") print(f"Interest rate coverage: {quality.interest_rate_coverage:.0%}") print(f"Debt investments: {quality.debt_count}") print(f"Equity investments: {quality.equity_count}")` **Note:** Not all BDCs provide detailed XBRL data for individual investments. Use `has_detailed_investments()` to check: `if arcc.has_detailed_investments(): investments = arcc.portfolio_investments() else: print("This BDC only provides aggregate data")` Cross-BDC Analysis ------------------ Use SEC DERA bulk datasets for analysis across all BDCs. ### Fetch Quarterly Data `from edgar.bdc import fetch_bdc_dataset dataset = fetch_bdc_dataset(2024, 3) print(f"Period: {dataset.period}") print(f"BDCs: {dataset.num_companies}") print(f"SOI entries: {dataset.num_soi_entries:,}")` **Output:** `Period: 2024Q3 BDCs: 148 SOI entries: 106,715` ### Search for Portfolio Companies Find which BDCs hold a specific private company: `soi = dataset.schedule_of_investments results = soi.search("Ivy Hill") print(results)` **Output:** `company bdc_name bdc_cik fair_value 0 Ivy Hill Asset Management, L.P. ARES CAPITAL CORP 1287750 1915300000.0` ### Find Most Common Holdings Identify the most widely-held private companies: `top = soi.top_companies(10) print(top)` **Output:** `company num_bdcs total_fair_value bdc_names 0 OA Buyer, Inc. 6 322406000.0 BARINGS BDC, BARINGS CAPITAL... 1 MRI Software LLC 5 287500000.0 ARES CAPITAL, GOLUB CAPITAL...` ### Subset by BDC Get data for a specific BDC from the bulk dataset: `# By CIK arcc_soi = soi[1287750] print(f"ARCC entries: {len(arcc_soi)}") # By BDCEntity arcc = bdcs.get_by_ticker("ARCC") arcc_soi = soi[arcc]` ### Industry Analysis Analyze industry concentration: `summary = dataset.summary_by_industry() print(summary.head(10))` Integration with Company Objects -------------------------------- BDC entities connect to standard Company functionality: `arcc = bdcs.get_by_ticker("ARCC") # Get the Company object company = arcc.get_company() print(company) # Access filings filings = arcc.get_filings(form="10-K") latest_10k = filings[0] print(f"Latest 10-K: {latest_10k.filing_date}") # Get the Schedule of Investments statement soi_statement = arcc.schedule_of_investments() print(soi_statement)` Common Use Cases ---------------- ### Private Company Research Find all BDC exposure to a private company: `from edgar.bdc import fetch_bdc_dataset dataset = fetch_bdc_dataset(2024, 3) soi = dataset.schedule_of_investments # Search for the company results = soi.search("MRI Software") print(f"Found in {len(results)} BDC positions") print(f"Total exposure: ${results['fair_value'].sum():,.0f}") print(f"BDCs holding: {results['bdc_name'].unique().tolist()}")` ### BDC Portfolio Comparison Compare portfolio composition across BDCs: `from edgar.bdc import get_bdc_list bdcs = get_bdc_list() tickers = ["ARCC", "MAIN", "GBDC"] for ticker in tickers: bdc = bdcs.get_by_ticker(ticker) if bdc and bdc.has_detailed_investments(): inv = bdc.portfolio_investments() quality = inv.data_quality print(f"{ticker}: {len(inv)} positions, " f"{quality.debt_count} debt, {quality.equity_count} equity")` ### Yield Analysis Analyze interest rates across a BDC's debt portfolio: `arcc = bdcs.get_by_ticker("ARCC") investments = arcc.portfolio_investments() # Filter to debt with interest rates df = investments.to_dataframe() debt = df[df['interest_rate'].notna()] print(f"Average interest rate: {debt['interest_rate'].mean():.2%}") print(f"Rate range: {debt['interest_rate'].min():.2%} - {debt['interest_rate'].max():.2%}")` Data Sources ------------ The BDC module uses two SEC data sources: | Source | Content | Best For | | --- | --- | --- | | **SEC BDC Report** | List of all BDCs with file numbers | Finding and identifying BDCs | | **DERA Quarterly Datasets** | Pre-extracted Schedule of Investments | Cross-BDC analysis | | **Individual 10-K/10-Q** | Detailed XBRL investment data | Deep dive into single BDC | Performance Tips ---------------- 1. **Cache the BDC list**: `get_bdc_list()` is cached after first call 2. **Use bulk datasets for cross-BDC analysis**: Much faster than parsing individual filings 3. **Check data availability**: Use `has_detailed_investments()` before parsing 4. **Filter early**: Use the `filter()` method to reduce data before analysis Next Steps ---------- Now that you can access BDC data, learn how to: * **[Extract Financial Statements](https://edgartools.readthedocs.io/en/stable/guides/extract-statements/) ** - Get balance sheets and income statements * **[Query XBRL Facts](https://edgartools.readthedocs.io/en/stable/xbrl-querying/) ** - Deep dive into XBRL data * **[Company Facts API](https://edgartools.readthedocs.io/en/stable/guides/company-facts/) ** - Historical financial metrics Related Documentation --------------------- * **[Company Subsets](https://edgartools.readthedocs.io/en/stable/company-subsets/) ** - Create groups of companies for analysis * **[Finding Companies](https://edgartools.readthedocs.io/en/stable/guides/finding-companies/) ** - General company lookup Back to top --- # Insider Trades (Form 4) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/insider-filings/#insider-trades-track-sec-form-4-insider-buying-and-selling-with-python) Insider Trades: Track SEC Form 4 Insider Buying and Selling with Python ======================================================================= Know when insiders buy or sell their own company's stock. SEC Forms 3, 4, and 5 disclose every transaction by officers, directors, and major shareholders. EdgarTools parses these filings into structured Python objects with computed insights like net position change and trading plan detection. `from edgar import Company snow = Company("SNOW") filing = snow.get_filings(form=4).latest(1) form4 = filing.obj() form4` ![Form 4 insider trade parsed with Python edgartools showing Snowflake director sale](https://edgartools.readthedocs.io/en/latest/images/snow-form4.webp) Three lines to see who traded, what they traded, and the net impact on their position. > **[See Snowflake's insider trading activity on edgar.tools — 186K+ filings pre-parsed →](https://app.edgar.tools/companies/SNOW?utm_source=edgartools-docs&utm_medium=see-live&utm_content=insider-filings) > ** * * * Get the Transaction Summary --------------------------- The `get_ownership_summary()` method returns a `TransactionSummary` with computed properties that answer the questions you actually care about. `summary = form4.get_ownership_summary() summary.insider_name # "Bruce I. Sachs" summary.position # "Director" summary.primary_activity # "Purchase", "Sale", "Option Exercise", etc. summary.net_change # 15000 (positive = bought, negative = sold) summary.net_value # 3260400.0 (net dollar value of trades) summary.remaining_shares # 36599` | Property | What it tells you | | --- | --- | | `primary_activity` | One-word categorization: Purchase, Sale, Tax Withholding, Grant/Award, Option Exercise, Mixed | | `net_change` | Net shares bought minus sold -- the single most important number | | `net_value` | Net dollar value of all transactions | | `remaining_shares` | Insider's position after all transactions | | `transaction_types` | List of unique activity types in this filing | | `has_non_derivatives` | Whether any common stock was traded | * * * Detect Automated Trading Plans ------------------------------ The `has_10b5_1_plan` property tells you whether trades were pre-scheduled under a Rule 10b5-1 plan. This matters because pre-scheduled sales are less informative than discretionary ones. `summary.has_10b5_1_plan # True, False, or None # True = trade executed under a 10b5-1 plan (automated, less signal) # False = footnotes exist but no plan mentioned (discretionary) # None = no footnotes available` EdgarTools detects this by scanning transaction footnotes for 10b5-1 references -- something that would require manual XML parsing otherwise. * * * Access Individual Transactions ------------------------------ For transaction-level detail, use the filtered DataFrame properties on the `Form4` object itself. `form4.market_trades # All open-market buys and sells form4.common_stock_purchases # Just the buys form4.common_stock_sales # Just the sells form4.shares_traded # Total shares across all market trades` The `market_trades` DataFrame includes these columns: | Column | What it is | | --- | --- | | `Date` | Transaction date | | `Security` | Security title | | `Shares` | Number of shares | | `Price` | Price per share | | `Remaining` | Shares owned after this transaction | | `AcquiredDisposed` | `"A"` (acquired) or `"D"` (disposed) | | `Code` | Transaction code (`P` = purchase, `S` = sale) | * * * Track Option Exercises and Derivatives -------------------------------------- Insider filings often include option exercises, RSU conversions, and other derivative transactions. `form4.option_exercises # Transactions with exercise code form4.derivative_trades # All derivative transactions` The derivative table includes exercise price, expiration date, and underlying security information. * * * Convert to DataFrame -------------------- The `to_dataframe()` method gives you full control over output format. ### Detailed view (one row per transaction) `df = form4.to_dataframe()` ### Summary view (one row per filing) `df = form4.to_dataframe(detailed=False)` This aggregates everything into a single row with computed columns like `Net Change`, `Net Value`, `Primary Activity`, and per-type breakdowns (`Purchase Shares`, `Avg Purchase Price`, `Sale Shares`, etc.). Useful for building datasets across thousands of filings. ### Strip metadata `df = form4.to_dataframe(include_metadata=False)` Removes the filing-level columns (`Date`, `Form`, `Issuer`, `Ticker`, `Insider`, `Position`) when you only need transaction data. * * * Initial Ownership (Form 3) -------------------------- When an insider first joins a company, they file a Form 3 disclosing what they already own. EdgarTools parses these into the same object hierarchy. `filing = Company("HROW").get_filings(form=3).latest(1) form3 = filing.obj() summary = form3.get_ownership_summary() # Returns InitialOwnershipSummary summary.total_shares # Total non-derivative shares owned summary.has_derivatives # True if they hold options/warrants summary.holdings # List of SecurityHolding objects` ![Form 3 initial beneficial ownership parsed with Python edgartools showing Harrow insider holdings](https://edgartools.readthedocs.io/en/latest/images/hrow-form3.webp) Each `SecurityHolding` in the list has: | Property | What it is | | --- | --- | | `security_title` | Name of the security | | `shares` | Number of shares or units | | `direct_ownership` | `True` if directly owned | | `ownership_description` | "Direct" or "Indirect (reason)" | | `is_derivative` | Whether this is a derivative holding | | `exercise_price` | Exercise price (derivatives only) | | `expiration_date` | Expiration date (derivatives only) | * * * Look Up a Specific Insider -------------------------- `from edgar import Company apple = Company("AAPL") # All insider filings (Forms 3, 4, 5) filings = apple.get_filings(form=[3, 4, 5]) # Just Form 4s form4_filings = apple.get_filings(form=4) # Latest transaction latest = form4_filings.latest(1).obj() print(f"{latest.insider_name} ({latest.position}): {latest.get_ownership_summary().primary_activity}")` * * * Common Analysis Patterns ------------------------ ### Find large purchases `from edgar import get_filings filings = get_filings(form=4) for f in filings[:20]: form4 = f.obj() if form4: summary = form4.get_ownership_summary() if summary.net_change > 10000: print(f"{summary.insider_name} bought {summary.net_change:,} shares of {summary.issuer}")` ### Filter out automated sales `summary = form4.get_ownership_summary() if summary.has_10b5_1_plan is False: # Discretionary trade -- potentially more informative print(f"{summary.primary_activity}: {summary.net_change:,} shares")` ### Build a dataset across filings `import pandas as pd filings = Company("AAPL").get_filings(form=4) rows = [] for f in filings[:50]: form4 = f.obj() if form4: rows.append(form4.to_dataframe(detailed=False)) df = pd.concat(rows, ignore_index=True)` See this on edgar.tools The code above parses individual Form 4 filings. **edgar.tools** connects 186K+ insider filings and 802K+ transactions into a searchable intelligence layer with sentiment analysis. * **[See Apple's insider trading activity →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=insider) ** * **[See Tesla's insider transactions →](https://app.edgar.tools/companies/TSLA?utm_source=edgartools-docs&utm_medium=see-live&utm_content=insider) ** Includes net buy/sell sentiment, executive profiles, and cross-filing linkages to 8-K material events. Free tier available. [Pricing →](https://app.edgar.tools/pricing?utm_source=edgartools-docs&utm_medium=see-live&utm_content=insider) * * * Metadata Quick Reference ------------------------ | Property | Returns | Example | | --- | --- | --- | | `form` | Form type | `"4"` | | `reporting_period` | Transaction date | `"2024-01-18"` | | `insider_name` | Reporting insider | `"Bruce I. Sachs"` | | `position` | Role at company | `"Director"` | | `issuer.name` | Company name | `"VERTEX PHARMACEUTICALS INC"` | | `issuer.ticker` | Ticker symbol | `"VRTX"` | | `issuer.cik` | Company CIK | `"875320"` | | `no_securities` | No securities owned | `False` | | `remarks` | Filing remarks | `""` | | `shares_traded` | Total shares in market trades | `15000` | * * * Methods Quick Reference ----------------------- | Call | Returns | What it does | | --- | --- | --- | | `form4.get_ownership_summary()` | `TransactionSummary` or `InitialOwnershipSummary` | Computed summary with net change, activity type, 10b5-1 detection | | `form4.get_transaction_activities()` | `list[TransactionActivity]` | All transactions as structured objects | | `form4.to_dataframe()` | `DataFrame` | Full transaction data, one row per trade | | `form4.to_dataframe(detailed=False)` | `DataFrame` | Single summary row with aggregated metrics | | `form4.market_trades` | `DataFrame` | Open-market buys and sells only | | `form4.common_stock_purchases` | `DataFrame` | Filtered to acquisitions | | `form4.common_stock_sales` | `DataFrame` | Filtered to dispositions | | `form4.option_exercises` | `DataFrame` | Option exercise transactions | | `form4.derivative_trades` | `DataHolder` | All derivative transactions | | `form4.extract_form3_holdings()` | `list[SecurityHolding]` | Holdings from Form 3 filings | | `form4.to_html()` | `str` | HTML representation | * * * Things to Know -------------- **Form 3 vs 4 vs 5.** Form 3 is initial ownership (when someone becomes an insider). Form 4 is changes (buys, sells, grants). Form 5 is an annual catch-up for anything not reported on Form 4. Most analysis focuses on Form 4. **Transaction codes.** `P` = open-market purchase, `S` = open-market sale, `M` = option exercise, `A` = grant/award, `F` = tax withholding, `G` = gift, `C` = conversion. The `primary_activity` property translates these for you. **Footnotes contain critical context.** Prices, share counts, and 10b5-1 plan disclosures often live in footnotes, not the transaction table. EdgarTools resolves footnote references automatically. **Derivative transactions are complex.** Option exercises often pair with a same-day sale. The `derivative_trades` property keeps these separate from common stock transactions. **Amended filings (3/A, 4/A, 5/A).** EdgarTools handles amendments transparently -- they parse identically to the original form types. * * * Related ------- * [Track Company Insiders](https://edgartools.readthedocs.io/en/latest/guides/company-insiders/) -- monitor insider activity for a specific company * [Working with Filings](https://edgartools.readthedocs.io/en/latest/guides/working-with-filing/) -- general filing access patterns Back to top --- # Understanding 10-K, 10-Q, and 8-K Report Objects in Python - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/concepts/data-objects/#understanding-10-k-10-q-and-8-k-report-objects-in-python) Understanding 10-K, 10-Q, and 8-K Report Objects in Python ========================================================== Introduction ------------ One of the most powerful features of edgartools is its Data Objects system. This system transforms raw SEC filing data into structured, easy-to-use Python objects that expose filing-specific properties and methods. Instead of dealing with complex HTML, XML, or XBRL parsing yourself, Data Objects handle all the heavy lifting, allowing you to focus on analysis rather than data extraction. This guide explains the conceptual framework behind Data Objects, how they work under the hood, and how to leverage them effectively in your SEC data analysis workflows. The Problem Data Objects Solve ------------------------------ SEC filings are notoriously complex documents: * They contain a mix of structured and unstructured data * They use different formats (HTML, XML, XBRL) depending on filing type and date * Their structure evolves over time as SEC requirements change * They often contain inconsistencies in formatting and organization * They require domain knowledge to interpret correctly Without Data Objects, working with SEC filings would require: 1. Downloading raw filing documents 2. Writing custom parsers for each filing type 3. Handling edge cases and inconsistencies 4. Extracting and organizing the data manually 5. Converting data into usable formats for analysis Data Objects eliminate these challenges by providing a consistent, intuitive interface to SEC filing data, regardless of the underlying format or structure. The Data Objects Architecture ----------------------------- ### Core Principles The Data Objects system is built on several key principles: 1. **Type-Specific Interfaces**: Each filing type has its own specialized interface that exposes only the relevant properties and methods. 2. **Lazy Parsing**: Content is parsed on-demand to minimize memory usage and processing time. 3. **Consistent Access Patterns**: Similar data is accessed through consistent patterns across different filing types. 4. **Rich Metadata**: Each object includes metadata about the filing, such as dates, filer information, and document structure. 5. **Transformation Capabilities**: Data can be easily transformed into formats like pandas DataFrames for analysis. ### Object Hierarchy Data Objects follow a hierarchical structure: `Filing (base class) ├── CompanyFiling │ ├── TenK (10-K Annual Report) │ ├── TenQ (10-Q Quarterly Report) │ └── EightK (8-K Current Report) ├── OwnershipFiling │ ├── Form3 (Initial Ownership) │ ├── Form4 (Changes in Ownership) │ └── Form5 (Annual Ownership Summary) ├── InvestmentFiling │ └── ThirteenF (13F Holdings Report) └── Other specialized filing types` Each object in this hierarchy inherits common functionality while adding specialized features for its filing type. How Data Objects Work --------------------- ### The Creation Process When you call the `.obj()` method on a Filing object, the following process occurs: 1. **Filing Type Detection**: The system identifies the filing type based on the form type and content. 2. **Parser Selection**: The appropriate parser is selected for that filing type. 3. **Object Instantiation**: A new Data Object of the correct type is created. 4. **Initial Parsing**: Basic metadata is parsed immediately. 5. **Lazy Loading Setup**: More complex content is set up for on-demand parsing. ### Parsing Strategies Data Objects use different parsing strategies depending on the filing type: * **HTML Parsing**: For narrative sections like business descriptions and risk factors * **XML Parsing**: For structured data like ownership transactions and fund holdings * **XBRL Processing**: For financial statements and other tagged financial data * **Table Extraction**: For tabular data embedded in filings * **Text Processing**: For extracting plain text from complex HTML structures These strategies are applied automatically based on the content being accessed. Working with Data Objects ------------------------- ### Common Patterns Across all Data Objects, you'll find these common patterns: 1. **Property Access**: Access filing sections or data through properties (e.g., `tenk.risk_factors`, `tenk.auditor`, `tenk.subsidiaries`, `tenk.reports`) 2. **Method Calls**: Perform operations on the data (e.g., `form4.get_net_shares_traded()`) 3. **Dictionary-Like Access**: Access specific items by key (e.g., `eightk["Item 2.01"]`) 4. **Iteration**: Iterate over collections within the filing (e.g., `for holding in thirteen_f.infotable`) 5. **Conversion**: Transform data into other formats (e.g., `balance_sheet.to_dataframe()`) ### Object Persistence Data Objects are designed to be lightweight and don't persist the entire filing content in memory. Instead, they: 1. Store references to the original filing content 2. Parse specific sections only when accessed 3. Cache parsed results to avoid repeated parsing 4. Release memory when no longer needed This approach allows you to work with very large filings efficiently. Advanced Usage Patterns ----------------------- ### Combining Multiple Data Objects You can combine data from multiple Data Objects for more sophisticated analysis: `# Compare financial data across quarters company = Company("AAPL") filings = company.get_filings(form=["10-K", "10-Q"]).head(5) data_objects = [filing.obj() for filing in filings] # Extract revenue from each filing revenues = [] for obj in data_objects: if hasattr(obj, "income_statement"): period_end = obj.period_end_date revenue = obj.income_statement.get_value("Revenues") revenues.append((period_end, revenue)) # Sort by date and analyze trend revenues.sort(key=lambda x: x[0])` See it live on edgar.tools The code above combines multiple Data Objects for cross-period analysis. **edgar.tools** does this automatically — multi-year financials, disclosure timelines, and filing comparisons for any company, with export to Excel or PDF. * **[See Apple's financials across multiple years →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=data-objects-concepts) ** * **[Browse 12 XBRL disclosure topics →](https://app.edgar.tools/disclosures?utm_source=edgartools-docs&utm_medium=see-live&utm_content=data-objects-concepts) ** Free tier available. Also includes a REST API and hosted MCP server for AI integrations. ### Custom Data Extraction You can extend Data Objects with your own extraction logic: `def extract_cybersecurity_risks(tenk): """Extract cybersecurity-related content from risk factors.""" if not hasattr(tenk, "risk_factors"): return None risk_text = tenk.risk_factors cyber_keywords = ["cyber", "hack", "breach", "data security", "privacy"] # Find paragraphs containing cyber keywords paragraphs = risk_text.split("\n\n") cyber_paragraphs = [p for p in paragraphs if any(k in p.lower() for k in cyber_keywords)] return cyber_paragraphs # Apply to a 10-K tenk = company.latest("10-K").obj() cyber_risks = extract_cybersecurity_risks(tenk)` ### Batch Processing For processing many filings efficiently: `# Process all 8-Ks from the past year company = Company("MSFT") filings = company.get_filings(form="8-K", start_date="2024-01-01") # Extract all press releases all_press_releases = [] for filing in filings: try: eightk = filing.obj() if eightk.has_press_release: for pr in eightk.press_releases: all_press_releases.append({ "date": eightk.date_of_report, "title": pr.title, "content": pr.content }) except Exception as e: print(f"Error processing filing {filing.accession_number}: {e}") print(f"Found {len(all_press_releases)} press releases")` Common Challenges and Solutions ------------------------------- ### Challenge: Handling Missing Data Not all filings contain all expected sections or data points: `# Safe access pattern tenk = filing.obj() if hasattr(tenk, "risk_factors") and tenk.risk_factors: # Process risk factors pass else: print("No risk factors section found") # For financial data try: revenue = income_stmt.get_value("Revenues") except ValueError: revenue = income_stmt.get_value("RevenueFromContractWithCustomerExcludingAssessedTax") except: revenue = None` ### Challenge: Handling Format Changes SEC filing formats evolve over time: `# Version-aware code tenk = filing.obj() filing_year = tenk.period_end_date.year if filing_year >= 2021: # Use newer XBRL taxonomy concepts revenue = income_stmt.get_value("RevenueFromContractWithCustomerExcludingAssessedTax") else: # Use older concepts revenue = income_stmt.get_value("Revenues")` ### Challenge: Processing Large Filings Some filings (especially 10-Ks) can be very large: `# Memory-efficient processing tenk = filing.obj() # Process one section at a time sections = ["business", "risk_factors", "management_discussion"] for section_name in sections: if hasattr(tenk, section_name): section = getattr(tenk, section_name) # Process section # ... # Explicitly delete to free memory del section` Best Practices -------------- ### 1\. Use the Right Object for the Task Choose the most specific Data Object for your needs: * Use `TenK`/`TenQ` for financial statement analysis * Use `TenK` for auditor info (`tenk.auditor`), subsidiaries (`tenk.subsidiaries`), and XBRL report pages (`tenk.reports`) * Use `EightK` for event monitoring * Use `Form4` for insider trading analysis * Use `ThirteenF` for fund holdings analysis ### 2\. Leverage Built-in Methods Data Objects include many helpful methods that save you from writing custom code: `# Instead of parsing manually: form4 = filing.obj() net_shares = form4.get_net_shares_traded() # Built-in method # Instead of calculating manually: thirteen_f = filing.obj() top_10 = thirteen_f.get_top_holdings(10) # Built-in method` ### 3\. Handle Errors Gracefully SEC filings can have inconsistencies that cause parsing errors: `try: data_obj = filing.obj() # Work with the object except Exception as e: print(f"Error parsing filing {filing.accession_number}: {e}") # Fall back to simpler access methods text = filing.text` ### 4\. Use Local Storage * Data Objects parse filing content on-demand * Large filings (like 10-Ks) may take a few seconds to parse * Consider using local storage for batch processing Conclusion ---------- Data Objects are the heart of edgartools' power and usability. By abstracting away the complexities of SEC filing formats and structures, they allow you to focus on analysis rather than data extraction. Understanding how Data Objects work and how to use them effectively will help you build more powerful, efficient, and maintainable SEC data analysis workflows. Whether you're analyzing financial statements, tracking insider trading, or researching investment funds, Data Objects provide a consistent, intuitive interface that makes working with SEC data a breeze. Additional Resources -------------------- * [Working with Financial Statements](https://edgartools.readthedocs.io/en/stable/guides/extract-statements/) * [Current Events (8-K)](https://edgartools.readthedocs.io/en/stable/eightk-filings/) * [Analyzing Insider Trading](https://edgartools.readthedocs.io/en/stable/guides/track-form4/) * [Institutional Holdings (13F)](https://edgartools.readthedocs.io/en/stable/guides/thirteenf-data-object-guide/) Back to top --- # Filter by Criteria - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/filtering-filings/#filter-sec-filings-by-form-type-date-ticker-exchange-and-cik) Filter SEC Filings — By Form Type, Date, Ticker, Exchange, and CIK ================================================================== Learn how to filter SEC filings by multiple criteria to find exactly what you need. Two Ways to Filter ------------------ You can filter filings in two ways: 1. **Filter while getting** - Use `get_filings()` parameters to filter from all SEC filings 2. **Filter after getting** - Use `.filter()` method to refine an existing `Filings` collection Both approaches work similarly, but filtering while getting is more efficient when you know the criteria upfront. Filter While Getting Filings ---------------------------- ### Filter by Form Type Get filings of a specific SEC form: `from edgar import get_filings # Single form type tenk = get_filings(2024, 1, form="10-K") # Multiple form types financial = get_filings(2024, 1, form=["10-K", "10-Q"]) # Proxy statements proxies = get_filings(2024, 1, form="DEF 14A")` ### Include or Exclude Amendments By default, amendments are included. Exclude them with `amendments=False`: `# Include amendments (default) all_10k = get_filings(2024, 1, form="10-K", amendments=True) # Exclude amendments original_only = get_filings(2024, 1, form="10-K", amendments=False)` ### Filter by Date #### Specific Date `# Filings on a specific date jan_15 = get_filings(2024, 1, filing_date="2024-01-15")` #### Date Range `# Filings between two dates jan_filings = get_filings(2024, 1, filing_date="2024-01-01:2024-01-31") # Q1 2024 q1 = get_filings(2024, filing_date="2024-01-01:2024-03-31")` #### Open-Ended Ranges `# From date onwards recent = get_filings(2024, 1, filing_date="2024-01-15:") # Up to a date older = get_filings(2024, 1, filing_date=":2024-01-15")` ### Combine Filters `# 10-K filings from January 2024, no amendments filings = get_filings( year=2024, quarter=1, form="10-K", filing_date="2024-01-01:2024-01-31", amendments=False )` See it live on edgar.tools The code above filters filings by form, date, and amendments. **edgar.tools** provides the same filtering in a visual interface — combine criteria and see results update in real time. * **[Filter the real-time filing stream →](https://app.edgar.tools/filings?utm_source=edgartools-docs&utm_medium=see-live&utm_content=filtering-filings) ** * **[Browse Apple's filing history →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=filtering-filings) ** Also available via REST API with form type, date range, and ticker filters. Free tier: 100 API calls/day. Filter After Getting Filings ---------------------------- Use the `.filter()` method to refine an existing collection: ### Filter by Form `filings = get_filings(2024, 1) # Filter to 10-K only tenk = filings.filter(form="10-K") # Multiple forms financial = filings.filter(form=["10-K", "10-Q"])` ### Filter by Date `filings = get_filings(2024, 1) # Specific date jan_1 = filings.filter(date="2024-01-01") # Date range jan_range = filings.filter(date="2024-01-01:2024-01-31") # From date onwards recent = filings.filter(date="2024-01-15:")` ### Filter by Company (CIK) `filings = get_filings(2024, 1) # Filter by CIK (integer) apple = filings.filter(cik=320193) # Filter by CIK (string) apple = filings.filter(cik="0000320193") # Multiple companies faang = filings.filter(cik=[320193, 1318605, 1652044])` ### Filter by Ticker `filings = get_filings(2024, 1) # Single ticker apple = filings.filter(ticker="AAPL") # Multiple tickers tech = filings.filter(ticker=["AAPL", "MSFT", "GOOGL", "AMZN"])` **Note:** Ticker filtering performs a CIK lookup first. If you know the CIK, use it directly for better performance. ### Filter by Exchange `filings = get_filings(2024, 1) # Single exchange nasdaq = filings.filter(exchange="NASDAQ") # Multiple exchanges major = filings.filter(exchange=["NASDAQ", "NYSE"])` **Available exchanges:** - NASDAQ - NYSE - CBOE - OTC ### Filter by Accession Number `filings = get_filings(2024, 1) # Single accession number filing = filings.filter(accession_number="0000320193-24-000001") # Multiple accession numbers specific = filings.filter(accession_number=[ "0000320193-24-000001", "0001318605-24-000001" ])` ### Filter Amendments `filings = get_filings(2024, 1, form="10-K") # Exclude amendments original_only = filings.filter(amendments=False) # Only amendments amendments_only = filings.filter(amendments=True)` Chain Filters ------------- Build complex queries by chaining multiple filters: `from edgar import get_filings # Start with all Q1 2024 filings filings = get_filings(2024, 1) # Chain filters for specificity result = (filings .filter(form="10-K") .filter(exchange="NASDAQ") .filter(date="2024-01-01:2024-01-31") .filter(amendments=False)) print(f"Found {len(result)} filings matching all criteria")` Alternatively, combine multiple criteria in one filter: `result = filings.filter( form="10-K", exchange="NASDAQ", date="2024-01-01:2024-01-31", amendments=False )` Use head, tail, and sample -------------------------- Limit results after filtering: ### head() Get the first n filings: `filings = get_filings(2024, 1, form="10-K") # Get first 10 first_10 = filings.head(10)` ### tail() Get the last n filings: `# Get last 10 last_10 = filings.tail(10)` ### sample() Get a random sample: `# Get random sample of 10 random_10 = filings.sample(10)` ### latest() Get most recent filings: `# Get latest single filing latest = filings.latest() # Get latest 20 filings latest_20 = filings.latest(20)` Search by Company Name ---------------------- Use `.find()` to search by company name: `filings = get_filings(2024, 1) # Find companies with "Technology" in name tech = filings.find("Technology") # Find specific company apple = filings.find("Apple") # Case-insensitive partial match results = filings.find("tesla")` Common Filtering Patterns ------------------------- ### Get Latest 10-K for NASDAQ Companies `from edgar import get_filings filings = get_filings(2024, 1, form="10-K") nasdaq = filings.filter(exchange="NASDAQ") latest_20 = nasdaq.latest(20) for filing in latest_20: print(f"{filing.company}: {filing.filing_date}")` ### Get All 8-K Filings for Specific Companies `filings = get_filings(2024, 1, form="8-K") # Filter to FAANG companies faang = filings.filter(ticker=["AAPL", "AMZN", "NFLX", "GOOGL", "META"]) print(f"Found {len(faang)} 8-K filings from FAANG")` ### Get Financial Reports from Tech Companies in January `# Get all Q1 filings filings = get_filings(2024, 1) # Filter to financial reports financial = filings.filter(form=["10-K", "10-Q"]) # Filter to NASDAQ (proxy for tech-heavy) nasdaq = financial.filter(exchange="NASDAQ") # Filter to January only jan = nasdaq.filter(date="2024-01-01:2024-01-31") print(f"Found {len(jan)} NASDAQ financial reports in January")` ### Get Original 10-K Filings (No Amendments) `filings = get_filings(2024, 1, form="10-K", amendments=False) # Or filter an existing collection all_10k = get_filings(2024, 1, form="10-K") original = all_10k.filter(amendments=False)` ### Get Filings by Year and Quarter Combinations `# Single year, single quarter q1_2024 = get_filings(2024, 1) # Single year, multiple quarters h1_2024 = get_filings(2024, [1, 2]) # Multiple years, single quarter q4_multi_year = get_filings([2022, 2023, 2024], 4) # Multiple years, all quarters multi_year = get_filings([2022, 2023, 2024]) # Year range range_2020_2024 = get_filings(range(2020, 2025)) # 2020-2024` Export Filtered Results ----------------------- ### To DataFrame `filings = get_filings(2024, 1, form="10-K") nasdaq = filings.filter(exchange="NASDAQ") # Convert to DataFrame df = nasdaq.to_pandas() # Or select specific columns df = nasdaq.to_pandas('company', 'filing_date', 'cik', 'accession_no') print(df.head())` ### To Parquet `filings = get_filings(2024, 1, form="10-K") nasdaq = filings.filter(exchange="NASDAQ") # Save as parquet nasdaq.save_parquet("nasdaq_10k_q1_2024.parquet")` Performance Tips ---------------- ### Filter Early **Efficient:** `# Filter using get_filings parameters filings = get_filings(2024, 1, form="10-K")` **Less Efficient:** `# Get everything then filter filings = get_filings(2024, 1).filter(form="10-K")` ### Use CIK Instead of Ticker **Efficient:** `# Filter by CIK (direct lookup) filings = filings.filter(cik=320193)` **Less Efficient:** `# Filter by ticker (requires CIK lookup first) filings = filings.filter(ticker="AAPL")` ### Limit Results Early `# Get only what you need filings = get_filings(2024, 1, form="10-K").head(50) # Better than processing all then limiting all_filings = get_filings(2024, 1, form="10-K") # ... process all ... limited = all_filings.head(50)` Error Handling -------------- `from edgar import get_filings try: filings = get_filings(2024, 1, form="10-K") if filings.empty: print("No filings found") else: # Filter nasdaq = filings.filter(exchange="NASDAQ") if nasdaq.empty: print("No NASDAQ filings") else: print(f"Found {len(nasdaq)} NASDAQ 10-K filings") except Exception as e: print(f"Error: {e}")` See Also -------- * **[Filings API Reference](https://edgartools.readthedocs.io/en/latest/api/filings/) ** - Complete Filings class documentation * **[Filing API Reference](https://edgartools.readthedocs.io/en/latest/api/filing/) ** - Individual filing operations * **[Search Filings Guide](https://edgartools.readthedocs.io/en/latest/guides/searching-filings/) ** - Finding specific filings * **[Current Filings Guide](https://edgartools.readthedocs.io/en/latest/guides/current-filings/) ** - Access today's filings * **[Working with Filings](https://edgartools.readthedocs.io/en/latest/guides/working-with-filing/) ** - Extract data from filings Back to top --- # Overview - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/data-objects/#sec-filing-data-objects-parsed-python-objects-for-every-form-type) SEC Filing Data Objects: Parsed Python Objects for Every Form Type ================================================================== Every SEC filing can be parsed into a structured Python object with one call: `filing.obj() # returns a TenK, EightK, ThirteenF, etc.` Browse the filing types below to find what you need. * * * Fund Entities ------------- Look up mutual funds and ETFs by ticker, series ID, or CIK. Navigate fund hierarchies and access portfolio reports. `from edgar import Fund, find_funds fund = Fund("VFINX") # Ticker, series ID, or CIK fund.get_portfolio() # Latest portfolio holdings` [Fund Entities guide](https://edgartools.readthedocs.io/en/stable/guides/fund-entity-guide/) * * * Annual & Quarterly Reports (10-K / 10-Q) ---------------------------------------- Read a company's financials, risk factors, and business description. `tenk = filing.obj() # TenK or TenQ tenk.income_statement # formatted financial statement tenk.risk_factors # full section text tenk.auditor # auditor name, PCAOB ID, location tenk.subsidiaries # subsidiaries from Exhibit 21 (10-K only) tenk.reports # XBRL viewer pages (statements, notes, details)` [Annual & Quarterly Reports](https://edgartools.readthedocs.io/en/stable/concepts/data-objects/) * * * Current Events (8-K) -------------------- Find out what just happened -- acquisitions, officer changes, earnings releases. `eightk = filing.obj() # EightK eightk.items # list of reported event codes eightk.press_releases # attached press releases` [Current Events guide](https://edgartools.readthedocs.io/en/stable/data-objects/guides/eightk-data-object-guide.md) * * * Insider Trades (Form 4) ----------------------- See who bought or sold shares and at what price. `form4 = filing.obj() # Ownership form4.reporting_owner # insider name form4.transactions # buy/sell details with prices` [Insider Trades guide](https://edgartools.readthedocs.io/en/stable/insider-filings/) See it live on edgar.tools Every filing type above — 10-K, 8-K, Form 4, 13F, proxy statements — is also browsable on **edgar.tools** with AI enrichment layered on top: * **[Browse Apple's filings, financials, and insider trades →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=data-objects) ** * **[Watch filings arrive in real time →](https://app.edgar.tools/filings?utm_source=edgartools-docs&utm_medium=see-live&utm_content=data-objects) ** * **[Search disclosures across 12 XBRL topics →](https://app.edgar.tools/disclosures?utm_source=edgartools-docs&utm_medium=see-live&utm_content=data-objects) ** Includes AI-classified 8-K events, insider sentiment analysis, and multi-year disclosure comparison. Free tier available. * * * Beneficial Ownership (Schedule 13D/G) ------------------------------------- Track activist investors and large institutional holders who own 5%+ of a company. `schedule = filing.obj() # Schedule13D or Schedule13G schedule.total_shares # aggregate beneficial ownership schedule.items.item4_purpose_of_transaction # activist intent (13D only)` [Beneficial Ownership guide](https://edgartools.readthedocs.io/en/stable/guides/schedule13dg-data-object-guide/) * * * Institutional Portfolios (13F) ------------------------------ Explore hedge fund and institutional investor holdings. `thirteenf = filing.obj() # ThirteenF thirteenf.infotable # full holdings table thirteenf.total_value # portfolio market value` [Institutional Portfolios guide](https://edgartools.readthedocs.io/en/stable/guides/thirteenf-data-object-guide/) * * * Proxy & Governance (DEF 14A) ---------------------------- Review executive compensation, board nominees, and shareholder proposals. `proxy = filing.obj() # ProxyStatement proxy.executive_compensation # pay tables proxy.proposals # shareholder vote items` [Proxy & Governance guide](https://edgartools.readthedocs.io/en/stable/guides/proxystatement-data-object-guide/) * * * Private Offerings (Form D) -------------------------- Track exempt securities offerings and the companies raising capital. `formd = filing.obj() # FormD formd.offering # offering details and amounts formd.recipients # related persons` [Private Offerings guide](https://edgartools.readthedocs.io/en/stable/guides/formd-data-object-guide/) * * * Crowdfunding Offerings (Form C) ------------------------------- Monitor crowdfunding campaigns under Regulation CF, including offering terms and issuer financials. `formc = filing.obj() # FormC formc.offering_information # target amount, deadline, securities formc.annual_report_disclosure # issuer financials (if C-AR)` [Crowdfunding guide](https://edgartools.readthedocs.io/en/stable/guides/formc-data-object-guide/) * * * Insider Sale Notices (Form 144) ------------------------------- Monitor planned insider sales before they happen. `form144 = filing.obj() # Form144 form144.proposed_sale_amount # shares to be sold form144.securities # security details` [Insider Sale Notices guide](https://edgartools.readthedocs.io/en/stable/guides/form144-data-object-guide/) * * * Fund Shareholder Reports (N-CSR / N-CSRS) ----------------------------------------- Parse certified annual and semiannual shareholder reports with expense ratios, performance data, and share class details. `report = filing.obj() # FundShareholderReport report.expense_data() # expense ratios per share class report.performance_data() # annual returns per share class` [Fund Shareholder Reports guide](https://edgartools.readthedocs.io/en/stable/guides/fundshareholderreport-data-object-guide/) * * * Fund Portfolio Holdings (NPORT-P) --------------------------------- Parse monthly mutual fund and ETF portfolio holdings -- every stock, bond, and derivative position. `report = filing.obj() # FundReport report.investment_data() # All portfolio positions as DataFrame` [Fund Portfolio Holdings guide](https://edgartools.readthedocs.io/en/stable/guides/nport-data-object-guide/) * * * Money Market Funds (N-MFP) -------------------------- Parse money market fund filings with portfolio holdings, yields, NAV, and liquidity metrics. `mmf = filing.obj() # MoneyMarketFund mmf.portfolio_data() # Securities sorted by market value` [Money Market Funds guide](https://edgartools.readthedocs.io/en/stable/guides/moneymarketfund-data-object-guide/) * * * Fund Census (N-CEN) ------------------- Parse annual fund census filings with series data, service providers, and ETF details. `census = filing.obj() # FundCensus census.series_data() # Fund series summary` [Fund Census guide](https://edgartools.readthedocs.io/en/stable/guides/fundcensus-data-object-guide/) * * * Fund Voting Records (N-PX) -------------------------- See how mutual funds voted on shareholder proposals. `npx = filing.obj() # FundReport npx.votes # vote records by proposal` [Fund Voting Records guide](https://edgartools.readthedocs.io/en/stable/guides/npx-data-object-guide/) * * * ABS Distribution Reports (Form 10-D) ------------------------------------ Extract structured CMBS loan and property data from asset-backed securities distribution reports. `ten_d = filing.obj() # TenD (CMBS only) ten_d.loans # loan-level DataFrame ten_d.properties # property-level DataFrame ten_d.asset_data.summary() # pool statistics` [ABS Distribution Reports guide](https://edgartools.readthedocs.io/en/stable/guides/tend-data-object-guide/) * * * Municipal Advisors (MA-I) ------------------------- Look up municipal advisor registrations and disciplinary history. `mai = filing.obj() # MunicipalAdvisorForm mai.advisor_name # advisor details` [Municipal Advisors guide](https://edgartools.readthedocs.io/en/stable/guides/mai-data-object-guide/) * * * Prospectus Supplements (424B) ----------------------------- Extract offering terms, pricing, underwriting, and dilution from shelf takedown prospectuses. `prospectus = filing.obj() # Prospectus424B deal = prospectus.deal # Deal: normalized deal summary deal.price # per-share price (float) deal.gross_proceeds # total offering amount deal.discount_rate # underwriting fee as fraction of price` [Prospectus Supplements guide](https://edgartools.readthedocs.io/en/stable/guides/prospectus424b-data-object-guide/) * * * How it works ------------ Call `filing.obj()` on any supported filing. EdgarTools detects the form type, parses the raw HTML/XML/XBRL, and returns the right data object. If a filing type isn't supported yet, you'll get an `UnsupportedFilingTypeError`. `from edgar import Company apple = Company("AAPL") filing = apple.get_latest_filing("10-K") tenk = filing.obj() # returns a TenK with all sections and financials` Back to top --- # Company Classification - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/company-classification/#company-classification) Company Classification ====================== EdgarTools automatically classifies every SEC entity across multiple dimensions: whether it is a domestic or foreign registrant, what kind of business it operates, and its regulatory filing status. These properties are derived from SEC data — SIC codes, state of incorporation, and filing history — so you rarely need to look anything up manually. Filer Type: Domestic, Foreign, or Canadian ------------------------------------------ The `filer_type` property tells you where a company is incorporated. This matters for understanding which annual report form the company files: domestic companies file 10-K, foreign private issuers file 20-F, and Canadian issuers file 40-F. `from edgar import Company Company("AAPL").filer_type # 'Domestic' Company("BABA").filer_type # 'Foreign' Company("CNQ").filer_type # 'Canadian'` The `is_foreign` convenience property returns `True` for both Foreign and Canadian filers: `Company("BABA").is_foreign # True Company("CNQ").is_foreign # True Company("AAPL").is_foreign # False` ### How filer type is determined EdgarTools uses a two-stage approach: 1. **State of incorporation** (preferred): The SEC stores a state or country code for each registered entity. A US state code means domestic; a country code from outside Canada means foreign; Canada codes mean Canadian. 2. **Filing history fallback**: When the state of incorporation is absent, EdgarTools inspects the entity's recent filings. A 40-F signals Canadian; a 20-F or 6-K signals foreign; a 10-K or 10-Q signals domestic. Extended fallbacks cover ADR deposit registrations (`F-6`), foreign registration statements (`F-1`, `F-3`), and domestic-only forms like Regulation Crowdfunding (`C`). Business Category ----------------- The `business_category` property classifies what kind of entity a company is. This is useful when building screens or analysis pipelines that should behave differently for, say, a bank versus a REIT versus an ordinary operating company. `from edgar import Company Company("AAPL").business_category # 'Operating Company' Company("AGNC").business_category # 'REIT' Company("JPM").business_category # 'Bank' Company("MET").business_category # 'Insurance Company' Company("ARCC").business_category # 'BDC'` ### Available categories | Category | Description | | --- | --- | | `Operating Company` | Standard corporation — the default for most SEC filers | | `REIT` | Real Estate Investment Trust (SIC 6798) | | `Bank` | Commercial banks and savings institutions | | `Insurance Company` | Life, casualty, title, and similar insurers | | `ETF` | Exchange-traded fund | | `Mutual Fund` | Open-end registered investment company | | `Closed-End Fund` | Closed-end registered investment company | | `BDC` | Business Development Company | | `Investment Manager` | Asset manager or institutional investment adviser | | `Holding Company` | Pure holding company (SIC 6719) | | `SPAC` | Blank check / special purpose acquisition company | | `Unknown` | Insufficient signals for classification | ### Convenience predicates Three boolean methods let you check the broad category without pattern-matching strings: `company = Company("AAPL") company.is_operating_company() # True company.is_fund() # False company.is_financial_institution() # False` `company = Company("JPM") company.is_operating_company() # False company.is_financial_institution() # True (Banks, Insurance, Investment Managers, BDCs)` `company = Company("SPY") company.is_fund() # True (ETF, Mutual Fund, Closed-End Fund)` ### How business category is determined Classification uses a priority chain: 1. **Definitive SIC codes**: SIC 6798 → REIT; SIC 6770 → SPAC; SIC 6021–6036 → Bank; SIC 6311–6371 → Insurance Company. 2. **Investment company forms**: Primary investment forms (`N-CSR`, `NPORT-P`) trigger fund classification; the name and entity type then distinguish ETF from Mutual Fund from Closed-End Fund. 3. **BDC signals**: Operating entities that file `N-2` forms or whose names contain "Capital Corp". 4. **Investment manager signals**: Entities with SIC 6211 or 6282, or that file `13F-HR`. 5. **Holding company**: SIC 6719. 6. **Default**: Operating Company. Filer Category: SEC Accelerated Filer Status -------------------------------------------- The SEC requires companies above certain public float thresholds to file on accelerated timelines. The `filer_category` property captures this classification. `from edgar import Company apple = Company("AAPL") apple.is_large_accelerated_filer # True (public float >= $700M) apple.is_accelerated_filer # False apple.is_smaller_reporting_company # False apple.is_emerging_growth_company # False` For smaller companies: `# A hypothetical small-cap company company = Company("BYFC") company.is_non_accelerated_filer # True (public float < $75M) company.is_smaller_reporting_company # True (public float < $250M or revenue < $100M)` ### Filer status thresholds | Status | Public Float | | --- | --- | | Large Accelerated Filer | \>= $700 million | | Accelerated Filer | \>= $75 million and < $700 million | | Non-Accelerated Filer | < $75 million | Two additional qualifications may apply alongside any base status: * **Smaller Reporting Company (SRC)**: Public float below $250 million, or annual revenue below $100 million with no public float above $700 million. SRCs may use scaled disclosure requirements. * **Emerging Growth Company (EGC)**: Revenue below $1.235 billion and IPO within the past five years. EGCs may defer certain accounting standards. For the full `FilerCategory` object with enum access: `from edgar import Company from edgar.enums import FilerStatus, FilerCategory category = Company("AAPL").filer_category category.status # FilerStatus.LARGE_ACCELERATED str(category) # 'Large accelerated filer' category.qualifications # [] category.is_smaller_reporting_company # False` Industry: SIC Code and Description ---------------------------------- Every SEC registrant is assigned a Standard Industrial Classification (SIC) code. EdgarTools exposes both the code and its human-readable description: `from edgar import Company apple = Company("AAPL") apple.sic # 3571 apple.industry # 'Electronic Computers' jpm = Company("JPM") jpm.sic # 6022 jpm.industry # 'State commercial banks-Federal Reserve members & state (non members)'` Entity vs. Individual --------------------- Not every SEC filer is a company. Insiders and beneficial owners file ownership forms (Forms 3, 4, 5 and Schedule 13D/G) as individuals. EdgarTools distinguishes these automatically: `from edgar import Company Company("AAPL").is_company # True Company("AAPL").is_individual # False` When you load an entity by CIK and that entity turns out to be a person rather than a company, `is_individual` returns `True`. This typically happens when looking up a CIK obtained from an ownership filing. Classification uses a nine-signal priority chain: exchange listings, state of incorporation, entity type from SEC data, filing history, EIN, and name keywords. Companies with tickers or a state of incorporation are definitively classified as companies. Filers with only insider ownership forms in their history are classified as individuals. Quick Reference --------------- | Property | Type | Returns | | --- | --- | --- | | `filer_type` | `str \\| None` | `'Domestic'`, `'Foreign'`, `'Canadian'`, or `None` | | `is_foreign` | `bool` | `True` for Foreign or Canadian registrants | | `business_category` | `str` | See business category table above | | `is_operating_company()` | `bool` | `True` for standard operating companies | | `is_fund()` | `bool` | `True` for ETF, Mutual Fund, or Closed-End Fund | | `is_financial_institution()` | `bool` | `True` for Bank, Insurance, Investment Manager, or BDC | | `sic` | `int \\| None` | Standard Industrial Classification code | | `industry` | `str \\| None` | SIC description | | `filer_category` | `FilerCategory` | Full parsed filer category object | | `is_large_accelerated_filer` | `bool` | Public float >= $700M | | `is_accelerated_filer` | `bool` | Public float >= $75M and < $700M | | `is_non_accelerated_filer` | `bool` | Public float < $75M | | `is_smaller_reporting_company` | `bool` | Qualifies as SRC | | `is_emerging_growth_company` | `bool` | Qualifies as EGC | | `is_company` | `bool` | `True` if the filer is a company | | `is_individual` | `bool` | `True` if the filer is a person | Related Guides -------------- * [Finding Companies](https://edgartools.readthedocs.io/en/stable/guides/finding-companies/) — Look up companies by ticker, CIK, or name * [Entity API Guide](https://edgartools.readthedocs.io/en/stable/guides/entity-api-guide/) — Filer category details and company icons * [BDC Guide](https://edgartools.readthedocs.io/en/stable/guides/bdc-guide/) — Working with Business Development Companies * [Fund Entity Guide](https://edgartools.readthedocs.io/en/stable/guides/fund-entity-guide/) — ETFs, mutual funds, and closed-end funds Back to top --- # Insider Trades (Form 4) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/insider-filings/#insider-trades-track-sec-form-4-insider-buying-and-selling-with-python) Insider Trades: Track SEC Form 4 Insider Buying and Selling with Python ======================================================================= Know when insiders buy or sell their own company's stock. SEC Forms 3, 4, and 5 disclose every transaction by officers, directors, and major shareholders. EdgarTools parses these filings into structured Python objects with computed insights like net position change and trading plan detection. `from edgar import Company snow = Company("SNOW") filing = snow.get_filings(form=4).latest(1) form4 = filing.obj() form4` ![Form 4 insider trade parsed with Python edgartools showing Snowflake director sale](https://edgartools.readthedocs.io/en/stable/images/snow-form4.webp) Three lines to see who traded, what they traded, and the net impact on their position. > **[See Snowflake's insider trading activity on edgar.tools — 186K+ filings pre-parsed →](https://app.edgar.tools/companies/SNOW?utm_source=edgartools-docs&utm_medium=see-live&utm_content=insider-filings) > ** * * * Get the Transaction Summary --------------------------- The `get_ownership_summary()` method returns a `TransactionSummary` with computed properties that answer the questions you actually care about. `summary = form4.get_ownership_summary() summary.insider_name # "Bruce I. Sachs" summary.position # "Director" summary.primary_activity # "Purchase", "Sale", "Option Exercise", etc. summary.net_change # 15000 (positive = bought, negative = sold) summary.net_value # 3260400.0 (net dollar value of trades) summary.remaining_shares # 36599` | Property | What it tells you | | --- | --- | | `primary_activity` | One-word categorization: Purchase, Sale, Tax Withholding, Grant/Award, Option Exercise, Mixed | | `net_change` | Net shares bought minus sold -- the single most important number | | `net_value` | Net dollar value of all transactions | | `remaining_shares` | Insider's position after all transactions | | `transaction_types` | List of unique activity types in this filing | | `has_non_derivatives` | Whether any common stock was traded | * * * Detect Automated Trading Plans ------------------------------ The `has_10b5_1_plan` property tells you whether trades were pre-scheduled under a Rule 10b5-1 plan. This matters because pre-scheduled sales are less informative than discretionary ones. `summary.has_10b5_1_plan # True, False, or None # True = trade executed under a 10b5-1 plan (automated, less signal) # False = footnotes exist but no plan mentioned (discretionary) # None = no footnotes available` EdgarTools detects this by scanning transaction footnotes for 10b5-1 references -- something that would require manual XML parsing otherwise. * * * Access Individual Transactions ------------------------------ For transaction-level detail, use the filtered DataFrame properties on the `Form4` object itself. `form4.market_trades # All open-market buys and sells form4.common_stock_purchases # Just the buys form4.common_stock_sales # Just the sells form4.shares_traded # Total shares across all market trades` The `market_trades` DataFrame includes these columns: | Column | What it is | | --- | --- | | `Date` | Transaction date | | `Security` | Security title | | `Shares` | Number of shares | | `Price` | Price per share | | `Remaining` | Shares owned after this transaction | | `AcquiredDisposed` | `"A"` (acquired) or `"D"` (disposed) | | `Code` | Transaction code (`P` = purchase, `S` = sale) | * * * Track Option Exercises and Derivatives -------------------------------------- Insider filings often include option exercises, RSU conversions, and other derivative transactions. `form4.option_exercises # Transactions with exercise code form4.derivative_trades # All derivative transactions` The derivative table includes exercise price, expiration date, and underlying security information. * * * Convert to DataFrame -------------------- The `to_dataframe()` method gives you full control over output format. ### Detailed view (one row per transaction) `df = form4.to_dataframe()` ### Summary view (one row per filing) `df = form4.to_dataframe(detailed=False)` This aggregates everything into a single row with computed columns like `Net Change`, `Net Value`, `Primary Activity`, and per-type breakdowns (`Purchase Shares`, `Avg Purchase Price`, `Sale Shares`, etc.). Useful for building datasets across thousands of filings. ### Strip metadata `df = form4.to_dataframe(include_metadata=False)` Removes the filing-level columns (`Date`, `Form`, `Issuer`, `Ticker`, `Insider`, `Position`) when you only need transaction data. * * * Initial Ownership (Form 3) -------------------------- When an insider first joins a company, they file a Form 3 disclosing what they already own. EdgarTools parses these into the same object hierarchy. `filing = Company("HROW").get_filings(form=3).latest(1) form3 = filing.obj() summary = form3.get_ownership_summary() # Returns InitialOwnershipSummary summary.total_shares # Total non-derivative shares owned summary.has_derivatives # True if they hold options/warrants summary.holdings # List of SecurityHolding objects` ![Form 3 initial beneficial ownership parsed with Python edgartools showing Harrow insider holdings](https://edgartools.readthedocs.io/en/stable/images/hrow-form3.webp) Each `SecurityHolding` in the list has: | Property | What it is | | --- | --- | | `security_title` | Name of the security | | `shares` | Number of shares or units | | `direct_ownership` | `True` if directly owned | | `ownership_description` | "Direct" or "Indirect (reason)" | | `is_derivative` | Whether this is a derivative holding | | `exercise_price` | Exercise price (derivatives only) | | `expiration_date` | Expiration date (derivatives only) | * * * Look Up a Specific Insider -------------------------- `from edgar import Company apple = Company("AAPL") # All insider filings (Forms 3, 4, 5) filings = apple.get_filings(form=[3, 4, 5]) # Just Form 4s form4_filings = apple.get_filings(form=4) # Latest transaction latest = form4_filings.latest(1).obj() print(f"{latest.insider_name} ({latest.position}): {latest.get_ownership_summary().primary_activity}")` * * * Common Analysis Patterns ------------------------ ### Find large purchases `from edgar import get_filings filings = get_filings(form=4) for f in filings[:20]: form4 = f.obj() if form4: summary = form4.get_ownership_summary() if summary.net_change > 10000: print(f"{summary.insider_name} bought {summary.net_change:,} shares of {summary.issuer}")` ### Filter out automated sales `summary = form4.get_ownership_summary() if summary.has_10b5_1_plan is False: # Discretionary trade -- potentially more informative print(f"{summary.primary_activity}: {summary.net_change:,} shares")` ### Build a dataset across filings `import pandas as pd filings = Company("AAPL").get_filings(form=4) rows = [] for f in filings[:50]: form4 = f.obj() if form4: rows.append(form4.to_dataframe(detailed=False)) df = pd.concat(rows, ignore_index=True)` See this on edgar.tools The code above parses individual Form 4 filings. **edgar.tools** connects 186K+ insider filings and 802K+ transactions into a searchable intelligence layer with sentiment analysis. * **[See Apple's insider trading activity →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=insider) ** * **[See Tesla's insider transactions →](https://app.edgar.tools/companies/TSLA?utm_source=edgartools-docs&utm_medium=see-live&utm_content=insider) ** Includes net buy/sell sentiment, executive profiles, and cross-filing linkages to 8-K material events. Free tier available. [Pricing →](https://app.edgar.tools/pricing?utm_source=edgartools-docs&utm_medium=see-live&utm_content=insider) * * * Metadata Quick Reference ------------------------ | Property | Returns | Example | | --- | --- | --- | | `form` | Form type | `"4"` | | `reporting_period` | Transaction date | `"2024-01-18"` | | `insider_name` | Reporting insider | `"Bruce I. Sachs"` | | `position` | Role at company | `"Director"` | | `issuer.name` | Company name | `"VERTEX PHARMACEUTICALS INC"` | | `issuer.ticker` | Ticker symbol | `"VRTX"` | | `issuer.cik` | Company CIK | `"875320"` | | `no_securities` | No securities owned | `False` | | `remarks` | Filing remarks | `""` | | `shares_traded` | Total shares in market trades | `15000` | * * * Methods Quick Reference ----------------------- | Call | Returns | What it does | | --- | --- | --- | | `form4.get_ownership_summary()` | `TransactionSummary` or `InitialOwnershipSummary` | Computed summary with net change, activity type, 10b5-1 detection | | `form4.get_transaction_activities()` | `list[TransactionActivity]` | All transactions as structured objects | | `form4.to_dataframe()` | `DataFrame` | Full transaction data, one row per trade | | `form4.to_dataframe(detailed=False)` | `DataFrame` | Single summary row with aggregated metrics | | `form4.market_trades` | `DataFrame` | Open-market buys and sells only | | `form4.common_stock_purchases` | `DataFrame` | Filtered to acquisitions | | `form4.common_stock_sales` | `DataFrame` | Filtered to dispositions | | `form4.option_exercises` | `DataFrame` | Option exercise transactions | | `form4.derivative_trades` | `DataHolder` | All derivative transactions | | `form4.extract_form3_holdings()` | `list[SecurityHolding]` | Holdings from Form 3 filings | | `form4.to_html()` | `str` | HTML representation | * * * Things to Know -------------- **Form 3 vs 4 vs 5.** Form 3 is initial ownership (when someone becomes an insider). Form 4 is changes (buys, sells, grants). Form 5 is an annual catch-up for anything not reported on Form 4. Most analysis focuses on Form 4. **Transaction codes.** `P` = open-market purchase, `S` = open-market sale, `M` = option exercise, `A` = grant/award, `F` = tax withholding, `G` = gift, `C` = conversion. The `primary_activity` property translates these for you. **Footnotes contain critical context.** Prices, share counts, and 10b5-1 plan disclosures often live in footnotes, not the transaction table. EdgarTools resolves footnote references automatically. **Derivative transactions are complex.** Option exercises often pair with a same-day sale. The `derivative_trades` property keeps these separate from common stock transactions. **Amended filings (3/A, 4/A, 5/A).** EdgarTools handles amendments transparently -- they parse identically to the original form types. * * * Related ------- * [Track Company Insiders](https://edgartools.readthedocs.io/en/stable/guides/company-insiders/) -- monitor insider activity for a specific company * [Working with Filings](https://edgartools.readthedocs.io/en/stable/guides/working-with-filing/) -- general filing access patterns Back to top --- # Search & Filter - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/searching-filings/#search-and-filter-sec-filings-by-form-date-and-company) Search and Filter SEC Filings by Form, Date, and Company ======================================================== Learn how to find the exact SEC filings you need using various search criteria and filtering methods. Prerequisites ------------- * Understanding of SEC filing types (10-K, 10-Q, 8-K, etc.) Basic Filing Search ------------------- ### Get Recent Filings Start with the most recent filings across all companies: `from edgar import get_filings # Get the 50 most recent filings recent_filings = get_filings() # Display basic information for filing in recent_filings[:5]: print(f"{filing.form}: {filing.company_name} ({filing.filing_date})")` **Output:** `10-Q: Apple Inc. (2024-05-02) 8-K: Microsoft Corporation (2024-05-01) 10-K: Amazon.com Inc (2024-04-30) 13F-HR: Berkshire Hathaway Inc (2024-04-29) 4: Tesla Inc (2024-04-28)` ### Search by Filing Type Find specific types of SEC forms: `# Get recent 10-K annual reports annual_reports = get_filings(form="10-K").head(20) # Get multiple form types quarterly_and_annual = get_filings(form=["10-K", "10-Q"]) # Exclude amendments (filings ending in /A) original_filings = get_filings(form="10-K", amendments=False).head(20) print(f"Found {len(annual_reports)} annual reports")` Search by Date Range -------------------- ### Specific Date `# Get all filings from a specific date filings_jan_1 = get_filings(filing_date="2024-01-01") print(f"Found {len(filings_jan_1)} filings on 2024-01-01")` ### Date Ranges `# Get filings from a date range q1_filings = get_filings(filing_date="2024-01-01:2024-03-31") # Get filings after a specific date recent_filings = get_filings(filing_date="2024-01-01:") # Get filings before a specific date older_filings = get_filings(filing_date=":2023-12-31") print(f"Q1 2024 filings: {len(q1_filings)}")` ### Year and Quarter Search Calendar Year vs Fiscal Year The `year` and `quarter` parameters refer to **when the filing was submitted to the SEC** (calendar year), **not** the fiscal year the filing covers. For example, a company with a fiscal year ending March 31, 2024 would file their annual 10-K in May or June 2024. Using `get_filings(2024)` would find this filing because it was **filed** in calendar year 2024, even though the 10-K covers fiscal year 2024. To find filings by fiscal year, use the company's `get_filings()` method and filter by the filing's fiscal period information available in the XBRL data. `# Get filings for entire calendar year (by filing date) filings_2023 = get_filings(2023) # Get filings for specific calendar quarter q4_2023 = get_filings(2023, 4) # Get multiple quarters q3_q4_2023 = get_filings(2023, [3, 4]) # Get multiple years multi_year = get_filings([2022, 2023]) # Get range of years (excludes end year) decade_filings = get_filings(range(2010, 2021)) print(f"2023 filings: {len(filings_2023)}") print(f"Q4 2023 filings: {len(q4_2023)}")` Company-Specific Filing Search ------------------------------ ### Get All Company Filings `from edgar import Company # Get all filings for a company apple = Company("AAPL") all_apple_filings = apple.get_filings() print(f"Apple has {len(all_apple_filings)} total filings")` ### Filter Company Filings `# Get specific form types for a company apple_10k = apple.get_filings(form="10-K") apple_quarterly = apple.get_filings(form=["10-Q", "10-K"]) # Get XBRL filings only apple_xbrl = apple.get_filings(is_xbrl=True) # Get inline XBRL filings apple_ixbrl = apple.get_filings(is_inline_xbrl=True) print(f"Apple 10-K filings: {len(apple_10k)}") print(f"Apple XBRL filings: {len(apple_xbrl)}")` ### Get Latest Filing `# Get the most recent filing of a specific type latest_10k = apple.get_filings(form="10-K").latest() latest_10q = apple.get_filings(form="10-Q").latest() print(f"Latest 10-K: {latest_10k.filing_date}") print(f"Latest 10-Q: {latest_10q.filing_date}") # Chain the calls for conciseness latest_annual = Company("MSFT").get_filings(form="10-K").latest()` See it live on edgar.tools The code above searches filings by form type, date, and company. **edgar.tools** puts the same search in a visual interface — filter by form, date range, and company with results updating in real time. * **[Browse the real-time filing stream →](https://app.edgar.tools/filings?utm_source=edgartools-docs&utm_medium=see-live&utm_content=searching-filings) ** * **[Search Apple's full filing history →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=searching-filings) ** Also includes a REST API with filing search endpoints. Free tier: 100 API calls/day. [Pricing →](https://app.edgar.tools/pricing?utm_source=edgartools-docs&utm_medium=see-live&utm_content=searching-filings) Advanced Filtering ------------------ ### Filter by Multiple Criteria `# Get Apple's 10-K filings from 2023 that are XBRL apple_filtered = apple.get_filings( form="10-K", is_xbrl=True ).filter(filing_date="2023-01-01:2023-12-31") print(f"Filtered results: {len(apple_filtered)}")` ### Filter by Accession Number `# Find specific filing by accession number specific_filing = apple.get_filings( accession_number="0000320193-23-000106" ) print(f"Found filing: {specific_filing[0].form}")` ### Filter by File Number `# Filter by SEC file number file_filtered = apple.get_filings( file_number="001-36743" ) print(f"Filings with file number: {len(file_filtered)}")` Cross-Company Search -------------------- ### Search by Industry `# Get recent filings and filter by company characteristics all_filings = get_filings() # Filter for technology companies (requires loading each company) tech_filings = [] for filing in all_filings[:100]: # Limit for performance try: company = Company(filing.cik) if "software" in company.industry.lower() or "computer" in company.industry.lower(): tech_filings.append(filing) except: continue print(f"Found {len(tech_filings)} filings from tech companies")` ### Search by Exchange `# Filter existing filings by exchange nasdaq_filings = all_filings.filter(exchange="NASDAQ") nyse_filings = all_filings.filter(exchange="NYSE") print(f"NASDAQ filings: {len(nasdaq_filings)}") print(f"NYSE filings: {len(nyse_filings)}")` ### Search by Ticker List `# Get filings for multiple specific companies tickers = ["AAPL", "MSFT", "GOOGL", "AMZN"] ticker_filings = all_filings.filter(ticker=tickers) print(f"Filings from specified tickers: {len(ticker_filings)}")` Specialized Filing Searches --------------------------- ### Insider Trading Filings `# Get recent insider trading filings insider_filings = get_filings(form=["3", "4", "5"]) print("Recent insider filings:") for filing in insider_filings[:10]: print(f" Form {filing.form}: {filing.company_name} ({filing.filing_date})")` ### Fund Holdings (13F) `# Get recent 13F filings (institutional investment managers) fund_filings = get_filings(form="13F-HR") print("Recent fund holdings filings:") for filing in fund_filings: print(f" {filing.company_name}: {filing.filing_date}")` ### Material Events (8-K) `# Get recent 8-K filings (material corporate events) event_filings = get_filings(form="8-K") print("Recent material events:") for filing in event_filings[:10]: print(f" {filing.company_name}: {filing.filing_date}")` ### IPO and Registration Statements `# Get S-1 filings (IPO registrations) ipo_filings = get_filings(form="S-1") print("Recent IPO filings:") for filing in ipo_filings: print(f" {filing.company_name}: {filing.filing_date}")` Working with Search Results --------------------------- ### Subset and Sample `filings = get_filings(form="10-K") # Get first 10 results first_ten = filings.head(10) # Get last 10 results last_ten = filings.tail(10) # Get random sample of 5 results random_sample = filings.sample(5) print(f"Total: {len(filings)}, Sample: {len(random_sample)}")` ### Convert to Pandas DataFrame `import pandas as pd # Convert filings to DataFrame for analysis filings_df = filings.to_pandas() # Analyze filing patterns filing_counts = filings_df.groupby(['form', 'company_name']).size() print("Filing counts by company and form:") print(filing_counts.head(10))` ### Access Underlying Data `# Access the PyArrow table directly import pyarrow as pa filings = get_filings(form="10-K") data_table: pa.Table = filings.data # Convert to Pandas for advanced analysis df = data_table.to_pandas() print(f"Columns available: {df.columns.tolist()}")` Performance Optimization ------------------------ ### Efficient Searching `# More efficient: Use specific parameters in get_filings() efficient = get_filings(form="10-K", filing_date="2023-01-01:") # Less efficient: Get all then filter inefficient = get_filings().filter(form="10-K").filter(filing_date="2023-01-01:") print(f"Efficient approach found: {len(efficient)} filings")` ### Caching Results `# Store frequently used searches apple = Company("AAPL") apple_10k_cache = apple.get_filings(form="10-K") # Reuse cached results for different analyses recent_10k = apple_10k_cache.head(5) oldest_10k = apple_10k_cache.tail(5)` Error Handling -------------- ### Handle Missing Data `try: filings = get_filings(form="INVALID-FORM") print(f"Found {len(filings)} filings") except Exception as e: print(f"Error searching filings: {e}")` ### Validate Search Results `filings = get_filings(form="10-K", limit=10) if len(filings) == 0: print("No filings found matching criteria") else: print(f"Found {len(filings)} filings") # Verify first result first_filing = filings[0] print(f"First result: {first_filing.form} from {first_filing.company_name}")` Common Search Patterns ---------------------- ### Earnings Season Analysis `# Find quarterly reports filed in typical earnings periods earnings_dates = [ "2024-01-15:2024-02-15", # Q4 earnings "2024-04-15:2024-05-15", # Q1 earnings "2024-07-15:2024-08-15", # Q2 earnings "2024-10-15:2024-11-15" # Q3 earnings ] earnings_filings = [] for date_range in earnings_dates: filings = get_filings(form="10-Q", filing_date=date_range) earnings_filings.extend(filings) print(f"Found {len(earnings_filings)} earnings period filings")` ### M&A Activity Monitoring `# Look for 8-K filings that might indicate M&A activity ma_filings = get_filings(form="8-K") # Filter for potential M&A keywords (requires examining filing content) potential_ma = [] for filing in ma_filings[:50]: # Limit for performance try: text = filing.text() if any(keyword in text.lower() for keyword in ['acquisition', 'merger', 'tender offer', 'purchase agreement']): potential_ma.append(filing) except: continue print(f"Found {len(potential_ma)} potential M&A filings")` Next Steps ---------- Now that you can search for filings effectively, learn how to: * **[Filter Filings by Date/Type](https://edgartools.readthedocs.io/en/stable/guides/filtering-filings/) ** - Advanced filtering techniques * **[Access Filing Attachments](https://edgartools.readthedocs.io/en/stable/guides/filing-attachments/) ** - Get supporting documents Related Documentation --------------------- * **[Filing API Reference](https://edgartools.readthedocs.io/en/stable/api/filing/) ** - Complete Filing class documentation * **[Filings API Reference](https://edgartools.readthedocs.io/en/stable/api/filings/) ** - Filings collection methods * **[Working with Filings](https://edgartools.readthedocs.io/en/stable/guides/working-with-filing/) ** - Original filing documentation Back to top --- # Beneficial Ownership (13D/G) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/schedule13dg-data-object-guide/#schedule-13dg-beneficial-ownership-reports) Schedule 13D/G: Beneficial Ownership Reports ============================================ Overview -------- **Schedule 13D** and **Schedule 13G** are SEC filings required when an investor acquires beneficial ownership of 5% or more of a company's voting securities. * **Schedule 13D** is filed by **active investors** -- those who may seek to influence or control the company (activist investors, acquirers). It requires detailed narrative disclosures about the purpose and intent of the investment. * **Schedule 13G** is a shorter form filed by **passive investors** -- institutional investors, mutual funds, or other holders with no intent to change or influence control. The `Schedule13D` and `Schedule13G` classes parse these XML filings into structured Python objects. Access Pattern -------------- `from edgar import Filing # Schedule 13D (activist investor) filing = Filing(form="SCHEDULE 13D", ...) schedule_13d = filing.obj() # Returns Schedule13D # Schedule 13G (passive investor) filing = Filing(form="SCHEDULE 13G", ...) schedule_13g = filing.obj() # Returns Schedule13G` You can also search for these filings by company or across all filers: `from edgar import Company, get_filings # Search by company company = Company("AAPL") filings_13d = company.get_filings(form="SCHEDULE 13D") filings_13g = company.get_filings(form="SCHEDULE 13G") # Or search across all recent filings recent_13d = get_filings(form="SCHEDULE 13D").head(10) recent_13g = get_filings(form="SCHEDULE 13G").head(10)` See it live on edgar.tools The code above parses individual 13D/G filings. **edgar.tools** connects beneficial ownership data into a searchable interface — see who holds 5%+ of any company and track activist investor positions. * **[See Apple's major holders →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=schedule13dg) ** * **[Browse Tesla's ownership filings →](https://app.edgar.tools/companies/TSLA?utm_source=edgartools-docs&utm_medium=see-live&utm_content=schedule13dg) ** Also includes insider trades, 13F institutional holdings, and cross-filing linkages. Free tier available. [Pricing →](https://app.edgar.tools/pricing?utm_source=edgartools-docs&utm_medium=see-live&utm_content=schedule13dg) * * * Schedule 13D (Active Investors) ------------------------------- When you call `filing.obj()` on a Schedule 13D filing, edgartools parses the XML structure and displays it as a rich, formatted panel: ![Schedule 13D Display](https://edgartools.readthedocs.io/en/latest/images/schedule13d-display.webp) ### Top-Level Properties | Property | Type | Description | | --- | --- | --- | | `issuer_info` | `IssuerInfo` | The company whose shares are being reported | | `security_info` | `SecurityInfo` | The class of securities (e.g., Common Stock) | | `reporting_persons` | `List[ReportingPerson]` | Beneficial owners filing the report | | `items` | `Schedule13DItems` | Items 1-7 narrative disclosures | | `signatures` | `List[Signature]` | Filing signatures | | `date_of_event` | `str` | Date that triggered the filing | | `previously_filed` | `bool` | Whether a prior filing exists | | `amendment_number` | `Optional[int]` | Amendment sequence number | | `is_amendment` | `bool` | Whether this is an amendment (`/A`) | | `filing_date` | `date` | Date filed with the SEC | | `total_shares` | `int` | Aggregate beneficial ownership (handles joint/separate filers) | | `total_percent` | `float` | Aggregate ownership percentage | ### Items (Narrative Disclosures) Schedule 13D requires detailed narrative responses to 7 items. **Item 4 (Purpose of Transaction)** is the most important -- it reveals the investor's intentions. | Property | Description | | --- | --- | | `item1_security_title` | Title of the security | | `item1_issuer_name` | Name of the issuer | | `item1_issuer_address` | Address of the issuer | | `item2_filing_persons` | Identity of reporting persons | | `item2_principal_occupation` | Occupation or business of filer | | `item2_citizenship` | Citizenship of filer | | `item3_source_of_funds` | Where the money came from | | **`item4_purpose_of_transaction`** | **Intent behind the investment** | | `item5_percentage_of_class` | Ownership percentage details | | `item5_number_of_shares` | Share count details | | `item5_transactions` | Recent transactions | | `item6_contracts` | Material contracts or arrangements | | `item7_exhibits` | Exhibits filed with the report | ### Example: Reading a Schedule 13D `from edgar import get_filings # Find a recent Schedule 13D filing filing = get_filings(form="SCHEDULE 13D").head(1)[0] schedule = filing.obj() # The rich panel is displayed automatically when you print the object print(schedule) # Issuer information print(schedule.issuer_info.name) # "ATLANTIC INTERNATIONAL CORP." print(schedule.issuer_info.cusip) # "048592109" # Ownership summary print(f"Total shares: {schedule.total_shares:,}") # 12,516,070 print(f"Ownership: {schedule.total_percent:.1f}%") # 16.7% # Reporting persons for person in schedule.reporting_persons: print(f"{person.name}: {person.aggregate_amount:,} shares ({person.percent_of_class}%)") # Output: # Guus Paul Wilhelm Franke: 12,516,070 shares (16.66%) # Axiom Partners GmbH: 0 shares (0.0%) # Purpose of transaction (activist intent) # This reveals what the investor plans to do purpose = schedule.items.item4_purpose_of_transaction print(purpose[:200]) # First 200 characters # "The information set forth in or incorporated by reference in Item 3 # and Item 6 of this Schedule 13D is hereby incorporated by reference..."` * * * Schedule 13G (Passive Investors) -------------------------------- When you call `filing.obj()` on a Schedule 13G filing, edgartools parses the XML structure and displays it as a clean, formatted panel. Notice how it's more concise than Schedule 13D, reflecting the passive nature of the investment: ![Schedule 13G Display](https://edgartools.readthedocs.io/en/latest/images/schedule13g-display.webp) ### Top-Level Properties | Property | Type | Description | | --- | --- | --- | | `issuer_info` | `IssuerInfo` | The company whose shares are being reported | | `security_info` | `SecurityInfo` | The class of securities | | `reporting_persons` | `List[ReportingPerson]` | Beneficial owners | | `items` | `Schedule13GItems` | Items 1-10 | | `signatures` | `List[Signature]` | Filing signatures | | `event_date` | `str` | Date that triggered the filing | | `rule_designation` | `Optional[str]` | SEC rule under which filing is made (e.g., "Rule 13d-1(c)") | | `is_amendment` | `bool` | Whether this is an amendment | | `filing_date` | `date` | Date filed with the SEC | | `total_shares` | `int` | Aggregate beneficial ownership | | `total_percent` | `float` | Aggregate ownership percentage | | `is_passive_investor` | `bool` | Always `True` for Schedule 13G | ### Example: Reading a Schedule 13G `from edgar import get_filings # Find a recent Schedule 13G filing filing = get_filings(form="SCHEDULE 13G").head(1)[0] schedule = filing.obj() # The rich panel is displayed automatically print(schedule) # Ownership summary print(schedule.issuer_info.name) # "BETA Technologies, Inc." print(f"Total shares: {schedule.total_shares:,}") # 11,753,896 print(f"Ownership: {schedule.total_percent:.1f}%") # 5.3% print(f"Rule: {schedule.rule_designation}") # "Rule 13d-1(d)" print(f"Passive: {schedule.is_passive_investor}") # True # Reporting persons with type for person in schedule.reporting_persons: print(f"{person.name} ({person.type_of_reporting_person}): " f"{person.aggregate_amount:,} shares") # Output: Amazon.com, Inc. (CO): 11,753,896 shares` * * * Practical Use Cases ------------------- ### Finding Activist Investors Schedule 13D filings reveal activist campaigns. You can monitor these to track potential corporate actions: `from edgar import get_filings # Get recent activist filings activist_filings = get_filings(form="SCHEDULE 13D").head(20) for filing in activist_filings: schedule = filing.obj() # Show high-ownership activist positions if schedule.total_percent > 10.0: print(f"{schedule.issuer_info.name}: {schedule.total_percent:.1f}%") # Check the purpose to understand their intent purpose_preview = schedule.items.item4_purpose_of_transaction[:200] print(f" Intent: {purpose_preview}...") print()` ### Tracking Institutional Ownership Schedule 13G filings show passive institutional holdings: `from edgar import Company # Track who owns significant stakes in a company company = Company("TSLA") institutional_filings = company.get_filings(form="SCHEDULE 13G") for filing in institutional_filings.head(10): schedule = filing.obj() for person in schedule.reporting_persons: print(f"{person.name}: {person.aggregate_amount:,} shares " f"({person.percent_of_class}%)")` * * * Shared Data Models ------------------ ### ReportingPerson Each reporting person (individual or entity) in the filing. | Property | Type | Description | | --- | --- | --- | | `cik` | `str` | SEC Central Index Key | | `name` | `str` | Person or entity name | | `citizenship` | `str` | Citizenship or place of organization | | `sole_voting_power` | `int` | Shares with sole voting authority | | `shared_voting_power` | `int` | Shares with shared voting authority | | `sole_dispositive_power` | `int` | Shares with sole dispositive authority | | `shared_dispositive_power` | `int` | Shares with shared dispositive authority | | `aggregate_amount` | `int` | Total shares beneficially owned | | `percent_of_class` | `float` | Percentage of outstanding shares | | `type_of_reporting_person` | `str` | Entity type code (e.g., "IN" = individual, "IA" = investment adviser) | | `member_of_group` | `Optional[str]` | `"a"` = joint filer, `"b"` = separate filer | | `total_voting_power` | `int` | Computed: sole + shared voting power | | `total_dispositive_power` | `int` | Computed: sole + shared dispositive power | ### IssuerInfo The company whose securities are being reported. | Property | Type | Description | | --- | --- | --- | | `cik` | `str` | Issuer's SEC CIK number | | `name` | `str` | Company name | | `cusip` | `str` | CUSIP identifier for the security | | `address` | `Optional[Address]` | Business address | ### SecurityInfo The class of securities subject to the filing. | Property | Type | Description | | --- | --- | --- | | `title` | `str` | Security title (e.g., "Common Stock") | | `cusip` | `str` | CUSIP identifier | ### Signature | Property | Type | Description | | --- | --- | --- | | `reporting_person` | `str` | Name of the reporting person | | `signature` | `str` | Signature as signed | | `title` | `str` | Title of signer | | `date` | `str` | Date signed | * * * Joint vs. Separate Filers ------------------------- When multiple reporting persons appear on a filing, edgartools automatically determines whether they are filing **jointly** (reporting the same shares) or **separately** (each holding distinct shares): * **Joint filers** (`member_of_group = "a"`): `total_shares` returns the shared amount, not a sum * **Hierarchical ownership** (parent-subsidiary chains): detected when percentages exceed 100%, takes the top-level amount * **Undeclared joint filers**: when all persons report identical share counts, treated as joint This means `total_shares` and `total_percent` always give you the correct aggregate, regardless of filing structure. * * * Tracking Amendments ------------------- Schedule 13D/G amendments (`/A` filings) report changes in ownership. You can compare amendments to track accumulation or liquidation: `from edgar.beneficial_ownership.amendments import OwnershipComparison original = Schedule13D.from_filing(original_filing) amendment = Schedule13D.from_filing(amended_filing) comparison = OwnershipComparison(current=amendment, previous=original) print(f"Shares changed: {comparison.shares_change:+,}") print(f"Percent changed: {comparison.percent_change:+.1f}%") print(f"Accumulating: {comparison.is_accumulating}") print(f"Liquidating: {comparison.is_liquidating}")` * * * Key Differences: Visual Comparison ---------------------------------- Notice the differences between the two screenshots above: **Schedule 13D (Activist)**: - Longer, more detailed panel with narrative disclosures - Includes "Purpose of Transaction" section explaining activist intent - Shows "Source of Funds" section - Multiple reporting persons common (individual + entity) - Event date shows when the triggering event occurred **Schedule 13G (Passive)**: - Cleaner, shorter panel reflecting passive nature - No narrative purpose section - Shows "Rule" designation (e.g., Rule 13d-1(d)) - Often a single institutional filer - Marks filing as "Passive Institutional Investor" * * * 13D vs. 13G: When to Use Which ------------------------------ | | Schedule 13D | Schedule 13G | | --- | --- | --- | | **Filer type** | Active / activist investors | Passive institutional investors | | **Intent** | May seek to influence or control | No intent to change control | | **Detail level** | 7 narrative items (purpose, source of funds, etc.) | 10 mostly procedural items | | **Key field** | `item4_purpose_of_transaction` | `rule_designation` | | **Filing deadline** | 10 days after crossing 5% | 45 days after calendar year-end | | **Amendment trigger** | Material changes | Annual or upon crossing thresholds | * * * Reporting Person Type Codes --------------------------- Common values for `type_of_reporting_person`: | Code | Description | | --- | --- | | `IN` | Individual | | `IA` | Investment Adviser | | `BD` | Broker-Dealer | | `BK` | Bank | | `IC` | Investment Company | | `HC` | Holding Company | | `CO` | Corporation | | `CP` | Co-Partnership | | `PN` | Pension Fund | | `OO` | Other | Back to top --- # Prospectus Supplements (424B) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/prospectus424b-data-object-guide/#prospectus-supplements-424b-parse-offering-terms-from-sec-filings) Prospectus Supplements (424B): Parse Offering Terms from SEC Filings ==================================================================== Overview -------- **424B filings** are prospectus supplements that companies file when they sell securities off a shelf registration (S-3/F-3). They contain the final deal terms: price, shares, proceeds, underwriting fees, and dilution impact. EdgarTools parses all 424B variants into a `Prospectus424B` object, with a `Deal` property that normalizes everything into clean numeric values. | Form | Typical Use | | --- | --- | | 424B2 | Structured notes, debt (large banks) | | 424B3 | Resale prospectuses (PIPE resales) | | 424B4 | Final priced prospectuses (IPOs) | | 424B5 | Shelf takedowns (ATM, firm commitment, PIPE) | Quick Start ----------- `from edgar import Company company = Company("ALZN") filing = company.get_filings(form="424B5")[0] prospectus = filing.obj() # Prospectus424B deal = prospectus.deal # Deal: normalized summary deal.price # 2.48 deal.shares # 1_500_000 deal.gross_proceeds # 3_720_000.0 deal.lead_bookrunner # "H.C. Wainwright & Co."` The Deal Object --------------- Access via `prospectus.deal`. Always returns a `Deal` object (never `None`). Individual properties return `None` when data is unavailable. ### Core Deal Terms | Property | Type | Description | | --- | --- | --- | | `price` | `float \\| None` | Per-unit offering price | | `shares` | `int \\| None` | Number of shares offered | | `gross_proceeds` | `float \\| None` | Total offering amount (before fees) | | `net_proceeds` | `float \\| None` | Proceeds after underwriting fees | | `security_type` | `str \\| None` | Security description ("Common Stock", "Senior Notes") | | `offering_type` | `OfferingType` | Enum: `FIRM_COMMITMENT`, `ATM`, `BEST_EFFORTS`, etc. | | `is_atm` | `bool` | Whether this is an at-the-market offering | ### Underwriting Economics | Property | Type | Description | | --- | --- | --- | | `fee_per_share` | `float \\| None` | Per-unit underwriting discount | | `total_fees` | `float \\| None` | Total underwriting fees | | `discount_rate` | `float \\| None` | Fee as fraction of price (0.05 = 5%) | | `fee_type` | `str \\| None` | `"underwriting_discount"` or `"placement_agent_fees"` | | `lead_bookrunner` | `str \\| None` | Lead underwriter or placement agent | | `underwriter_count` | `int` | Number of underwriters in syndicate | ### Dilution (Equity Offerings Only) | Property | Type | Description | | --- | --- | --- | | `dilution_per_share` | `float \\| None` | Dilution to new investors | | `dilution_pct` | `float \\| None` | Dilution as percentage | | `shares_before` | `int \\| None` | Shares outstanding before offering | | `shares_after` | `int \\| None` | Shares outstanding after offering | | `ntbv_before` | `float \\| None` | Net tangible book value per share before | | `ntbv_after` | `float \\| None` | Net tangible book value per share after | ### Serialization `deal.to_dict() # Flat dict of all non-None values (good for DataFrames) deal.to_context() # Markdown-KV text for LLM prompts` Offering Classification ----------------------- The `offering_type` property classifies the deal: | Value | Description | Price/Shares Available? | | --- | --- | --- | | `FIRM_COMMITMENT` | Bank buys all shares, resells | Yes | | `ATM` | At-the-market (sold gradually) | Usually no (market price) | | `BEST_EFFORTS` | Agent sells on best-efforts basis | Yes | | `PIPE_RESALE` | Resale of privately placed shares | Varies | | `STRUCTURED_NOTE` | Bank-issued structured product | Different meaning | | `DEBT_OFFERING` | Corporate bonds / notes | Usually percentage | `if deal.is_atm: # Price and shares are typically None for ATM offerings print(f"ATM program: up to ${deal.gross_proceeds:,.0f}") else: print(f"{deal.shares:,} shares @ ${deal.price:.2f}")` Prospectus Sub-Objects ---------------------- The `Prospectus424B` exposes the raw extracted data that the Deal synthesizes: `prospectus.cover_page # CoverPageData: company, registration, flags prospectus.pricing # PricingData: per-unit and total columns prospectus.underwriting # UnderwritingInfo: syndicate, fee type prospectus.offering_terms # OfferingTerms: shares, warrants, use of proceeds prospectus.selling_stockholders # SellingStockholdersData: PIPE resale tables prospectus.dilution # DilutionData: NTBV impact table prospectus.capitalization # CapitalizationData: actual vs. as-adjusted prospectus.structured_note_terms # StructuredNoteTerms: CUSIP, maturity (424B2) prospectus.filing_fees # FilingFeesData: from XBRL exhibit` Selling Stockholders (PIPE Resale Filings) ------------------------------------------ For PIPE resale prospectuses (typically 424B3), the selling stockholders table lists investors reselling privately placed shares: `ss = prospectus.selling_stockholders # SellingStockholdersData or None if ss: ss.count # Number of selling stockholders for entry in ss.stockholders: entry.name # "Lincoln Park Capital Fund, LLC" entry.shares # 1500000 (parsed int, None on failure) entry.shares_before # 2000000 entry.shares_after # 500000 entry.pct_before # 9.5 (parsed float) entry.pct_after # 2.8 entry.warrants # 750000 (warrants/convertibles, if present)` Raw string values are always preserved (`shares_offered`, `shares_before_offering`, etc.). The numeric properties (`shares`, `shares_before`, etc.) parse them to `int`/`float`, returning `None` on failure. ### DataFrame Output `df = ss.to_dataframe() # Returns DataFrame with numeric columns: # name | shares_before | pct_before | shares_offered | shares_after | pct_after | warrants` ### Offering Type Check `if prospectus.offering_type.has_selling_stockholders: # This is a PIPE_RESALE or BASE_PROSPECTUS_UPDATE ss = prospectus.selling_stockholders` Shelf Lifecycle --------------- Track where a prospectus sits in its shelf registration lifecycle: `lc = prospectus.lifecycle # ShelfLifecycle lc.takedown_number # 3 (this is the 3rd offering) lc.total_takedowns # 5 lc.shelf_expires # date(2027, 8, 2) lc.avg_days_between_takedowns # 180.0 lc.shelf_registration # Filing object for the S-3` Working with Multiple Offerings ------------------------------- Build a DataFrame of a company's offering history: `import pandas as pd from edgar import Company company = Company("ALZN") filings = company.get_filings(form="424B5") rows = [] for filing in filings: prospectus = filing.obj() d = prospectus.deal.to_dict() d['filing_date'] = str(filing.filing_date) rows.append(d) df = pd.DataFrame(rows)` Rich Display ------------ Both `Prospectus424B` and `Deal` render as Rich panels in terminals and notebooks: `prospectus # Shows cover page, pricing table, underwriting prospectus.deal # Compact deal summary panel` Back to top --- # Company Subsets - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/company-subsets/#company-subsets) Company Subsets =============== The `edgar.reference.company_subsets` module provides powerful and flexible tools for creating subsets of companies from SEC reference data. This is especially useful for research, analysis, educational purposes, and machine learning tasks where you need specific groups of companies. Key Features ------------ * **Exchange-based selection**: Filter by NYSE, NASDAQ, OTC, CBOE * **Popularity-based selection**: Get popular stocks, mega-cap companies, etc. * **Sampling capabilities**: Random sampling, stratified sampling, top N selection * **Filtering and combination utilities**: Include/exclude specific companies, combine sets * **Fluent interface**: Chain operations for readable, flexible subset creation * **Consistent output**: All functions return standardized DataFrames with `['cik', 'ticker', 'name', 'exchange']` columns Quick Start ----------- `from edgar.reference.company_subsets import ( CompanySubset, get_companies_by_exchanges, get_popular_companies, get_random_sample ) # Simple exchange-based selection nyse_companies = get_companies_by_exchanges('NYSE') print(f"Found {len(nyse_companies)} NYSE companies") # Get popular companies popular = get_popular_companies() print(f"Found {len(popular)} popular companies") # Random sampling random_100 = get_random_sample(n=100, random_state=42) print(f"Sampled {len(random_100)} random companies")` Fluent Interface with CompanySubset ----------------------------------- The `CompanySubset` class provides a powerful fluent interface for building complex company selections: `from edgar.reference.company_subsets import CompanySubset, PopularityTier # Complex selection with method chaining companies = (CompanySubset() .from_exchange(['NYSE', 'Nasdaq']) # Major exchanges only .exclude_tickers(['JPM', 'GS', 'C']) # Exclude some financials .sample(50, random_state=42) # Take random sample .get()) # Get the DataFrame print(f"Selected {len(companies)} companies") print(companies.head()) # Popular tech companies tech_subset = (CompanySubset() .from_popular(PopularityTier.POPULAR) # Popular companies .filter_by(lambda df: df['name'].str.contains('tech|software|computer', case=False)) .top(20, by='ticker') # Top 20 alphabetically .get())` See it live on edgar.tools The code above builds company subsets programmatically. **edgar.tools** lets you browse and search 940K+ SEC entities visually — filter by exchange, industry, or name. * **[Search the full SEC entity database →](https://app.edgar.tools/companies?utm_source=edgartools-docs&utm_medium=see-live&utm_content=company-subsets) ** * **[See Apple's company profile →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=company-subsets) ** Free tier available. Also includes a REST API for programmatic company lookups. [API docs →](https://app.edgar.tools/docs?utm_source=edgartools-docs&utm_medium=see-live&utm_content=company-subsets) Core Functions -------------- ### Exchange-Based Selection Filter companies by stock exchange: `from edgar.reference.company_subsets import get_companies_by_exchanges # Single exchange nyse_companies = get_companies_by_exchanges('NYSE') nasdaq_companies = get_companies_by_exchanges('Nasdaq') # Multiple exchanges major_exchanges = get_companies_by_exchanges(['NYSE', 'Nasdaq']) all_exchanges = get_companies_by_exchanges(['NYSE', 'Nasdaq', 'OTC', 'CBOE']) print(f"NYSE: {len(nyse_companies)} companies") print(f"NASDAQ: {len(nasdaq_companies)} companies") print(f"Major exchanges: {len(major_exchanges)} companies")` ### Popular Companies Access curated lists of popular and well-known companies: `from edgar.reference.company_subsets import get_popular_companies, PopularityTier # All popular companies all_popular = get_popular_companies() # By popularity tier mega_cap = get_popular_companies(PopularityTier.MEGA_CAP) # Top 10 popular = get_popular_companies(PopularityTier.POPULAR) # Top 50 mainstream = get_popular_companies(PopularityTier.MAINSTREAM) # Top 100 emerging = get_popular_companies(PopularityTier.EMERGING) # All available print(f"Mega cap: {len(mega_cap)} companies") print(f"Popular: {len(popular)} companies") print(f"All popular: {len(all_popular)} companies")` ### Sampling Methods Create representative samples from larger datasets: `from edgar.reference.company_subsets import ( get_random_sample, get_stratified_sample, get_top_companies_by_metric ) # Random sampling random_sample = get_random_sample(n=200, random_state=42) # Stratified sampling (maintains exchange proportions) stratified_sample = get_stratified_sample( n=100, stratify_by='exchange', random_state=42 ) # Top companies by name (alphabetical) top_alphabetical = get_top_companies_by_metric( n=50, metric='name', ascending=True ) # Sample from a specific subset nyse_random = get_random_sample( get_companies_by_exchanges('NYSE'), n=100, random_state=42 )` Filtering and Combining ----------------------- ### Include/Exclude Specific Companies `from edgar.reference.company_subsets import filter_companies, exclude_companies all_companies = get_all_companies() # Include specific tickers (FAANG companies) faang = filter_companies( all_companies, ticker_list=['META', 'AAPL', 'AMZN', 'NFLX', 'GOOGL'] ) # Include companies with names containing specific text tech_companies = filter_companies( all_companies, name_contains='Technology' ) # Include specific CIKs specific_companies = filter_companies( all_companies, cik_list=[320193, 1018724, 1652044] # AAPL, AMZN, GOOGL ) # Exclude financial companies (simplified example) non_financial = exclude_companies( all_companies, ticker_list=['JPM', 'GS', 'C', 'BAC', 'WFC'] ) # Exclude companies with 'Corp' in name non_corp = exclude_companies( all_companies, name_contains='Corp' )` ### Custom Filtering Apply custom filtering logic: `from edgar.reference.company_subsets import filter_companies # Custom filter function def large_company_filter(df): """Filter to companies with longer names (proxy for larger companies).""" return df[df['name'].str.len() > 20] # Apply custom filter large_companies = filter_companies( get_companies_by_exchanges('NYSE'), custom_filter=large_company_filter ) # Using lambda for simple filters short_tickers = filter_companies( get_popular_companies(), custom_filter=lambda df: df[df['ticker'].str.len() <= 4] )` ### Combining and Intersecting Sets `from edgar.reference.company_subsets import combine_company_sets, intersect_company_sets # Get different company sets nyse_companies = get_companies_by_exchanges('NYSE') popular_companies = get_popular_companies() tech_companies = filter_companies(get_all_companies(), name_contains='Tech') # Union: Combine multiple sets (removes duplicates) combined = combine_company_sets([nyse_companies, popular_companies, tech_companies]) # Intersection: Find companies present in all sets nyse_popular = intersect_company_sets([nyse_companies, popular_companies]) popular_tech = intersect_company_sets([popular_companies, tech_companies]) print(f"Combined: {len(combined)} companies") print(f"NYSE + Popular intersection: {len(nyse_popular)} companies") print(f"Popular + Tech intersection: {len(popular_tech)} companies")` Convenience Functions --------------------- Pre-defined functions for common company groupings: `from edgar.reference.company_subsets import ( get_faang_companies, get_tech_giants, get_dow_jones_sample ) # FAANG companies (Meta, Apple, Amazon, Netflix, Google) faang = get_faang_companies() # Major tech companies tech_giants = get_tech_giants() # Dow Jones Industrial Average sample dow_sample = get_dow_jones_sample() print(f"FAANG: {len(faang)} companies") print(f"Tech Giants: {len(tech_giants)} companies") print(f"Dow Sample: {len(dow_sample)} companies") # Display the companies print("\nFAANG Companies:") for _, company in faang.iterrows(): print(f" {company['ticker']}: {company['name']}")` Advanced Examples ----------------- ### Research Dataset Creation Create a balanced research dataset: `from edgar.reference.company_subsets import CompanySubset, PopularityTier # Create a research dataset with companies from different tiers research_dataset = [] # Get 20 mega-cap companies mega_cap = (CompanySubset() .from_popular(PopularityTier.MEGA_CAP) .sample(20, random_state=42) .get()) # Get 30 popular mid-tier companies mid_tier = (CompanySubset() .from_popular(PopularityTier.POPULAR) .exclude_tickers(mega_cap['ticker'].tolist()) # Don't overlap .sample(30, random_state=42) .get()) # Get 50 random companies from major exchanges random_companies = (CompanySubset() .from_exchange(['NYSE', 'Nasdaq']) .exclude_tickers(mega_cap['ticker'].tolist() + mid_tier['ticker'].tolist()) .sample(50, random_state=42) .get()) # Combine all for final research set research_companies = combine_company_sets([mega_cap, mid_tier, random_companies]) print(f"Research dataset: {len(research_companies)} companies") # Analyze composition exchange_dist = research_companies['exchange'].value_counts() print("\nExchange distribution:") print(exchange_dist)` ### Sector-Based Analysis Create industry-focused subsets: `# Create sector-based subsets (simplified approach using name patterns) sectors = { 'technology': ['tech', 'software', 'computer', 'digital'], 'financial': ['bank', 'financial', 'insurance', 'capital'], 'healthcare': ['health', 'medical', 'pharma', 'bio'], 'energy': ['energy', 'oil', 'gas', 'power'], 'retail': ['retail', 'store', 'market', 'shop'] } sector_companies = {} all_companies = get_companies_by_exchanges(['NYSE', 'Nasdaq']) for sector, keywords in sectors.items(): # Create pattern for all keywords pattern = '|'.join(keywords) sector_subset = filter_companies( all_companies, custom_filter=lambda df, p=pattern: df[df['name'].str.contains(p, case=False)] ) sector_companies[sector] = sector_subset print(f"{sector.title()}: {len(sector_subset)} companies") # Get top 10 from each sector for analysis analysis_set = [] for sector, companies in sector_companies.items(): top_10 = get_top_companies_by_metric(companies, n=10, metric='ticker') analysis_set.append(top_10) final_analysis_set = combine_company_sets(analysis_set) print(f"\nFinal analysis set: {len(final_analysis_set)} companies across sectors")` ### Machine Learning Dataset Preparation Prepare balanced datasets for ML training: `from edgar.reference.company_subsets import get_stratified_sample # Create training/test split with stratification all_popular = get_popular_companies() # Training set (70% of data, stratified by exchange) training_companies = get_stratified_sample( all_popular, n=int(len(all_popular) * 0.7), stratify_by='exchange', random_state=42 ) # Test set (remaining companies) test_companies = all_popular[~all_popular['cik'].isin(training_companies['cik'])] print(f"Training set: {len(training_companies)} companies") print(f"Test set: {len(test_companies)} companies") # Verify stratification worked print("\nTraining exchange distribution:") print(training_companies['exchange'].value_counts(normalize=True)) print("\nTest exchange distribution:") print(test_companies['exchange'].value_counts(normalize=True))` Data Structure -------------- All functions return a standardized pandas DataFrame with these columns: * **`cik`** (int): SEC Central Index Key - unique company identifier * **`ticker`** (str): Stock ticker symbol (e.g., 'AAPL', 'MSFT') * **`name`** (str): Official company name * **`exchange`** (str): Stock exchange ('NYSE', 'Nasdaq', 'OTC', 'CBOE', etc.) `# Example output structure companies = get_random_sample(5) print(companies) # cik ticker name exchange # 0 320193 AAPL Apple Inc. Nasdaq # 1 1018724 AMZN Amazon.com, Inc. Nasdaq # 2 1652044 GOOGL Alphabet Inc. Nasdaq # 3 789019 MSFT Microsoft Corporation Nasdaq # 4 1326801 META Meta Platforms, Inc Nasdaq` Error Handling -------------- The module includes robust error handling and logging: `# Functions gracefully handle errors and return empty DataFrames empty_result = get_companies_by_exchanges('INVALID_EXCHANGE') print(f"Invalid exchange result: {len(empty_result)} companies") # Check for empty results companies = get_random_sample(n=10) if companies.empty: print("No companies found") else: print(f"Found {len(companies)} companies") # All functions include logging for debugging import logging logging.basicConfig(level=logging.DEBUG) # Now function calls will show debug information companies = get_popular_companies()` Performance Considerations -------------------------- * **Caching**: `get_all_companies()` uses LRU cache for performance * **Lazy evaluation**: CompanySubset operations are efficient and don't duplicate data unnecessarily * **Memory efficient**: Functions work with DataFrame views when possible * **Batch operations**: Use combine/intersect functions instead of loops for better performance `# Efficient: Use batch operations company_sets = [ get_companies_by_exchanges('NYSE'), get_companies_by_exchanges('Nasdaq'), get_popular_companies() ] combined = combine_company_sets(company_sets) # Less efficient: Multiple individual operations in loops # combined = pd.DataFrame() # for exchange in ['NYSE', 'Nasdaq']: # exchange_companies = get_companies_by_exchanges(exchange) # combined = pd.concat([combined, exchange_companies]) # Avoid this pattern` Integration with Edgar Tools ---------------------------- Company subsets integrate seamlessly with other Edgar tools: `from edgar import Company from edgar.reference.company_subsets import get_tech_giants # Get tech companies and analyze their latest filings tech_companies = get_tech_giants() for _, company_info in tech_companies.head(5).iterrows(): try: company = Company(company_info['ticker']) latest_filing = company.get_filings(form='10-K').latest() print(f"{company_info['ticker']}: Latest 10-K filed {latest_filing.filing_date}") except: print(f"{company_info['ticker']}: No recent 10-K found")` Best Practices -------------- 1. **Use appropriate sample sizes**: Don't sample more companies than you need for analysis 2. **Set random seeds**: Use `random_state` parameter for reproducible results 3. **Handle empty results**: Always check if returned DataFrames are empty 4. **Combine operations efficiently**: Use method chaining with CompanySubset for readable code 5. **Cache results**: Store company subsets if you'll reuse them multiple times 6. **Validate data**: Check that your filters return expected results `# Good: Reproducible and efficient companies = (CompanySubset() .from_exchange('NYSE') .sample(100, random_state=42) .get()) # Store for reuse cached_companies = companies.copy() # Good: Check for empty results if not companies.empty: print(f"Analysis ready with {len(companies)} companies") else: print("No companies found matching criteria")` This module provides a comprehensive toolkit for creating company subsets tailored to your specific research, analysis, or educational needs. The combination of simple functions and the powerful fluent interface makes it easy to create both simple selections and complex, multi-criteria company datasets. Back to top --- # Beneficial Ownership (13D/G) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/schedule13dg-data-object-guide/#schedule-13dg-beneficial-ownership-reports) Schedule 13D/G: Beneficial Ownership Reports ============================================ Overview -------- **Schedule 13D** and **Schedule 13G** are SEC filings required when an investor acquires beneficial ownership of 5% or more of a company's voting securities. * **Schedule 13D** is filed by **active investors** -- those who may seek to influence or control the company (activist investors, acquirers). It requires detailed narrative disclosures about the purpose and intent of the investment. * **Schedule 13G** is a shorter form filed by **passive investors** -- institutional investors, mutual funds, or other holders with no intent to change or influence control. The `Schedule13D` and `Schedule13G` classes parse these XML filings into structured Python objects. Access Pattern -------------- `from edgar import Filing # Schedule 13D (activist investor) filing = Filing(form="SCHEDULE 13D", ...) schedule_13d = filing.obj() # Returns Schedule13D # Schedule 13G (passive investor) filing = Filing(form="SCHEDULE 13G", ...) schedule_13g = filing.obj() # Returns Schedule13G` You can also search for these filings by company or across all filers: `from edgar import Company, get_filings # Search by company company = Company("AAPL") filings_13d = company.get_filings(form="SCHEDULE 13D") filings_13g = company.get_filings(form="SCHEDULE 13G") # Or search across all recent filings recent_13d = get_filings(form="SCHEDULE 13D").head(10) recent_13g = get_filings(form="SCHEDULE 13G").head(10)` See it live on edgar.tools The code above parses individual 13D/G filings. **edgar.tools** connects beneficial ownership data into a searchable interface — see who holds 5%+ of any company and track activist investor positions. * **[See Apple's major holders →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=schedule13dg) ** * **[Browse Tesla's ownership filings →](https://app.edgar.tools/companies/TSLA?utm_source=edgartools-docs&utm_medium=see-live&utm_content=schedule13dg) ** Also includes insider trades, 13F institutional holdings, and cross-filing linkages. Free tier available. [Pricing →](https://app.edgar.tools/pricing?utm_source=edgartools-docs&utm_medium=see-live&utm_content=schedule13dg) * * * Schedule 13D (Active Investors) ------------------------------- When you call `filing.obj()` on a Schedule 13D filing, edgartools parses the XML structure and displays it as a rich, formatted panel: ![Schedule 13D Display](https://edgartools.readthedocs.io/en/stable/images/schedule13d-display.webp) ### Top-Level Properties | Property | Type | Description | | --- | --- | --- | | `issuer_info` | `IssuerInfo` | The company whose shares are being reported | | `security_info` | `SecurityInfo` | The class of securities (e.g., Common Stock) | | `reporting_persons` | `List[ReportingPerson]` | Beneficial owners filing the report | | `items` | `Schedule13DItems` | Items 1-7 narrative disclosures | | `signatures` | `List[Signature]` | Filing signatures | | `date_of_event` | `str` | Date that triggered the filing | | `previously_filed` | `bool` | Whether a prior filing exists | | `amendment_number` | `Optional[int]` | Amendment sequence number | | `is_amendment` | `bool` | Whether this is an amendment (`/A`) | | `filing_date` | `date` | Date filed with the SEC | | `total_shares` | `int` | Aggregate beneficial ownership (handles joint/separate filers) | | `total_percent` | `float` | Aggregate ownership percentage | ### Items (Narrative Disclosures) Schedule 13D requires detailed narrative responses to 7 items. **Item 4 (Purpose of Transaction)** is the most important -- it reveals the investor's intentions. | Property | Description | | --- | --- | | `item1_security_title` | Title of the security | | `item1_issuer_name` | Name of the issuer | | `item1_issuer_address` | Address of the issuer | | `item2_filing_persons` | Identity of reporting persons | | `item2_principal_occupation` | Occupation or business of filer | | `item2_citizenship` | Citizenship of filer | | `item3_source_of_funds` | Where the money came from | | **`item4_purpose_of_transaction`** | **Intent behind the investment** | | `item5_percentage_of_class` | Ownership percentage details | | `item5_number_of_shares` | Share count details | | `item5_transactions` | Recent transactions | | `item6_contracts` | Material contracts or arrangements | | `item7_exhibits` | Exhibits filed with the report | ### Example: Reading a Schedule 13D `from edgar import get_filings # Find a recent Schedule 13D filing filing = get_filings(form="SCHEDULE 13D").head(1)[0] schedule = filing.obj() # The rich panel is displayed automatically when you print the object print(schedule) # Issuer information print(schedule.issuer_info.name) # "ATLANTIC INTERNATIONAL CORP." print(schedule.issuer_info.cusip) # "048592109" # Ownership summary print(f"Total shares: {schedule.total_shares:,}") # 12,516,070 print(f"Ownership: {schedule.total_percent:.1f}%") # 16.7% # Reporting persons for person in schedule.reporting_persons: print(f"{person.name}: {person.aggregate_amount:,} shares ({person.percent_of_class}%)") # Output: # Guus Paul Wilhelm Franke: 12,516,070 shares (16.66%) # Axiom Partners GmbH: 0 shares (0.0%) # Purpose of transaction (activist intent) # This reveals what the investor plans to do purpose = schedule.items.item4_purpose_of_transaction print(purpose[:200]) # First 200 characters # "The information set forth in or incorporated by reference in Item 3 # and Item 6 of this Schedule 13D is hereby incorporated by reference..."` * * * Schedule 13G (Passive Investors) -------------------------------- When you call `filing.obj()` on a Schedule 13G filing, edgartools parses the XML structure and displays it as a clean, formatted panel. Notice how it's more concise than Schedule 13D, reflecting the passive nature of the investment: ![Schedule 13G Display](https://edgartools.readthedocs.io/en/stable/images/schedule13g-display.webp) ### Top-Level Properties | Property | Type | Description | | --- | --- | --- | | `issuer_info` | `IssuerInfo` | The company whose shares are being reported | | `security_info` | `SecurityInfo` | The class of securities | | `reporting_persons` | `List[ReportingPerson]` | Beneficial owners | | `items` | `Schedule13GItems` | Items 1-10 | | `signatures` | `List[Signature]` | Filing signatures | | `event_date` | `str` | Date that triggered the filing | | `rule_designation` | `Optional[str]` | SEC rule under which filing is made (e.g., "Rule 13d-1(c)") | | `is_amendment` | `bool` | Whether this is an amendment | | `filing_date` | `date` | Date filed with the SEC | | `total_shares` | `int` | Aggregate beneficial ownership | | `total_percent` | `float` | Aggregate ownership percentage | | `is_passive_investor` | `bool` | Always `True` for Schedule 13G | ### Example: Reading a Schedule 13G `from edgar import get_filings # Find a recent Schedule 13G filing filing = get_filings(form="SCHEDULE 13G").head(1)[0] schedule = filing.obj() # The rich panel is displayed automatically print(schedule) # Ownership summary print(schedule.issuer_info.name) # "BETA Technologies, Inc." print(f"Total shares: {schedule.total_shares:,}") # 11,753,896 print(f"Ownership: {schedule.total_percent:.1f}%") # 5.3% print(f"Rule: {schedule.rule_designation}") # "Rule 13d-1(d)" print(f"Passive: {schedule.is_passive_investor}") # True # Reporting persons with type for person in schedule.reporting_persons: print(f"{person.name} ({person.type_of_reporting_person}): " f"{person.aggregate_amount:,} shares") # Output: Amazon.com, Inc. (CO): 11,753,896 shares` * * * Practical Use Cases ------------------- ### Finding Activist Investors Schedule 13D filings reveal activist campaigns. You can monitor these to track potential corporate actions: `from edgar import get_filings # Get recent activist filings activist_filings = get_filings(form="SCHEDULE 13D").head(20) for filing in activist_filings: schedule = filing.obj() # Show high-ownership activist positions if schedule.total_percent > 10.0: print(f"{schedule.issuer_info.name}: {schedule.total_percent:.1f}%") # Check the purpose to understand their intent purpose_preview = schedule.items.item4_purpose_of_transaction[:200] print(f" Intent: {purpose_preview}...") print()` ### Tracking Institutional Ownership Schedule 13G filings show passive institutional holdings: `from edgar import Company # Track who owns significant stakes in a company company = Company("TSLA") institutional_filings = company.get_filings(form="SCHEDULE 13G") for filing in institutional_filings.head(10): schedule = filing.obj() for person in schedule.reporting_persons: print(f"{person.name}: {person.aggregate_amount:,} shares " f"({person.percent_of_class}%)")` * * * Shared Data Models ------------------ ### ReportingPerson Each reporting person (individual or entity) in the filing. | Property | Type | Description | | --- | --- | --- | | `cik` | `str` | SEC Central Index Key | | `name` | `str` | Person or entity name | | `citizenship` | `str` | Citizenship or place of organization | | `sole_voting_power` | `int` | Shares with sole voting authority | | `shared_voting_power` | `int` | Shares with shared voting authority | | `sole_dispositive_power` | `int` | Shares with sole dispositive authority | | `shared_dispositive_power` | `int` | Shares with shared dispositive authority | | `aggregate_amount` | `int` | Total shares beneficially owned | | `percent_of_class` | `float` | Percentage of outstanding shares | | `type_of_reporting_person` | `str` | Entity type code (e.g., "IN" = individual, "IA" = investment adviser) | | `member_of_group` | `Optional[str]` | `"a"` = joint filer, `"b"` = separate filer | | `total_voting_power` | `int` | Computed: sole + shared voting power | | `total_dispositive_power` | `int` | Computed: sole + shared dispositive power | ### IssuerInfo The company whose securities are being reported. | Property | Type | Description | | --- | --- | --- | | `cik` | `str` | Issuer's SEC CIK number | | `name` | `str` | Company name | | `cusip` | `str` | CUSIP identifier for the security | | `address` | `Optional[Address]` | Business address | ### SecurityInfo The class of securities subject to the filing. | Property | Type | Description | | --- | --- | --- | | `title` | `str` | Security title (e.g., "Common Stock") | | `cusip` | `str` | CUSIP identifier | ### Signature | Property | Type | Description | | --- | --- | --- | | `reporting_person` | `str` | Name of the reporting person | | `signature` | `str` | Signature as signed | | `title` | `str` | Title of signer | | `date` | `str` | Date signed | * * * Joint vs. Separate Filers ------------------------- When multiple reporting persons appear on a filing, edgartools automatically determines whether they are filing **jointly** (reporting the same shares) or **separately** (each holding distinct shares): * **Joint filers** (`member_of_group = "a"`): `total_shares` returns the shared amount, not a sum * **Hierarchical ownership** (parent-subsidiary chains): detected when percentages exceed 100%, takes the top-level amount * **Undeclared joint filers**: when all persons report identical share counts, treated as joint This means `total_shares` and `total_percent` always give you the correct aggregate, regardless of filing structure. * * * Tracking Amendments ------------------- Schedule 13D/G amendments (`/A` filings) report changes in ownership. You can compare amendments to track accumulation or liquidation: `from edgar.beneficial_ownership.amendments import OwnershipComparison original = Schedule13D.from_filing(original_filing) amendment = Schedule13D.from_filing(amended_filing) comparison = OwnershipComparison(current=amendment, previous=original) print(f"Shares changed: {comparison.shares_change:+,}") print(f"Percent changed: {comparison.percent_change:+.1f}%") print(f"Accumulating: {comparison.is_accumulating}") print(f"Liquidating: {comparison.is_liquidating}")` * * * Key Differences: Visual Comparison ---------------------------------- Notice the differences between the two screenshots above: **Schedule 13D (Activist)**: - Longer, more detailed panel with narrative disclosures - Includes "Purpose of Transaction" section explaining activist intent - Shows "Source of Funds" section - Multiple reporting persons common (individual + entity) - Event date shows when the triggering event occurred **Schedule 13G (Passive)**: - Cleaner, shorter panel reflecting passive nature - No narrative purpose section - Shows "Rule" designation (e.g., Rule 13d-1(d)) - Often a single institutional filer - Marks filing as "Passive Institutional Investor" * * * 13D vs. 13G: When to Use Which ------------------------------ | | Schedule 13D | Schedule 13G | | --- | --- | --- | | **Filer type** | Active / activist investors | Passive institutional investors | | **Intent** | May seek to influence or control | No intent to change control | | **Detail level** | 7 narrative items (purpose, source of funds, etc.) | 10 mostly procedural items | | **Key field** | `item4_purpose_of_transaction` | `rule_designation` | | **Filing deadline** | 10 days after crossing 5% | 45 days after calendar year-end | | **Amendment trigger** | Material changes | Annual or upon crossing thresholds | * * * Reporting Person Type Codes --------------------------- Common values for `type_of_reporting_person`: | Code | Description | | --- | --- | | `IN` | Individual | | `IA` | Investment Adviser | | `BD` | Broker-Dealer | | `BK` | Bank | | `IC` | Investment Company | | `HC` | Holding Company | | `CO` | Corporation | | `CP` | Co-Partnership | | `PN` | Pension Fund | | `OO` | Other | Back to top --- # Track Insider Trading: Analyze SEC Form 4 Buy and Sell Transactions - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/track-form4/#track-insider-trading-analyze-sec-form-4-buy-and-sell-transactions) Track Insider Trading: Analyze SEC Form 4 Buy and Sell Transactions =================================================================== Introduction ------------ Form 4 filings provide valuable insights into insider trading activity. When corporate insiders (directors, officers, or beneficial owners of more than 10% of a company's stock) buy or sell shares of their company, they must report these transactions to the SEC via Form 4 filings. These filings can reveal important signals about insiders' confidence in their company's future. edgartools makes it easy to retrieve, parse, and analyze Form 4 filings programmatically, allowing you to track insider trading patterns without manual effort. Understanding Form 4 Filings ---------------------------- Before diving into code, it's important to understand what Form 4 filings contain: * **Reporting Person Information**: Name, relationship to the company (e.g., CEO, Director) * **Transaction Details**: Date, type of security, number of shares, price per share * **Transaction Codes**: Codes that indicate the nature of the transaction (e.g., P for purchase, S for sale) * **Ownership Information**: Direct or indirect ownership, total shares held after transaction Retrieving Form 4 Filings ------------------------- ### By Company To retrieve Form 4 filings for a specific company: `from edgar import Company, get_filings # Using Company object company = Company("AAPL") form4_filings = company.get_filings(form="4") # Or using global get_filings form4_filings = get_filings(form="4", ticker="AAPL") # View the most recent filings recent_filings = form4_filings.head(5) for filing in recent_filings: print(f"Date: {filing.filing_date}, Person: {filing.reporting_owner_name}")` ### By Date Range To find Form 4 filings within a specific date range: `# Get Form 4 filings from Jan 1, 2024 to present form4_filings = get_filings( form="4", ticker="MSFT", start_date="2024-01-01", end_date="2024-07-01" ) print(f"Found {len(form4_filings)} Form 4 filings")` ### By Reporting Person To focus on a specific insider's activity: `form4_filings = get_filings(form="4", ticker="TSLA") # Filter by reporting person's name musk_filings = form4_filings.filter(reporting_owner_name="Musk Elon") print(f"Found {len(musk_filings)} Form 4 filings by Elon Musk")` Working with Form 4 Data Objects -------------------------------- edgartools provides a specialized `Form4` data object that makes it easy to access structured data from these filings: `# Get a specific Form 4 filing filing = form4_filings.latest() # Convert to Form4 data object form4 = filing.obj() # Access basic metadata print(f"Filing date: {form4.filing_date}") print(f"Reporting owner: {form4.reporting_owner_name}") print(f"Relationship: {form4.reporting_owner_relationship}") print(f"Company: {form4.issuer_name} ({form4.issuer_ticker})")` ### Accessing Transaction Details Form 4 filings can contain multiple transactions. Access them through the `transactions` property: `# Examine all transactions in the filing for i, transaction in enumerate(form4.transactions): print(f"\nTransaction {i+1}:") print(f"Date: {transaction.transaction_date}") print(f"Type: {transaction.transaction_code} ({transaction.get_transaction_code_description()})") print(f"Shares: {transaction.shares}") print(f"Price: ${transaction.price_per_share:.2f}") print(f"Value: ${transaction.value:.2f}") print(f"Direct/Indirect: {transaction.ownership}") print(f"Shares owned after: {transaction.shares_owned_following_transaction}")` ### Understanding Transaction Codes Form 4 transactions use codes to indicate different types of transactions: `# Common transaction codes and their meanings transaction_codes = { 'P': 'Open market or private purchase of securities', 'S': 'Open market or private sale of securities', 'A': 'Grant, award, or other acquisition', 'D': 'Disposition to the issuer (e.g., forfeiture, cancellation)', 'M': 'Exercise or conversion of derivative security', 'G': 'Gift', 'V': 'Voluntary transaction with issuer' } # Check what type of transaction this is for transaction in form4.transactions: code = transaction.transaction_code description = transaction_codes.get(code, "Other transaction type") print(f"Transaction code {code}: {description}") print(f"Shares: {transaction.shares}")` Analyzing Insider Transactions ------------------------------ ### Calculating Net Shares Traded Calculate whether an insider is buying or selling on net: `# Calculate net shares traded in a filing net_shares = form4.get_net_shares_traded() if net_shares > 0: print(f"Insider BOUGHT a net {net_shares:,} shares") elif net_shares < 0: print(f"Insider SOLD a net {abs(net_shares):,} shares") else: print("Insider had no net change in position")` ### Aggregating Transactions by Company Track recent insider activity for a company: `import pandas as pd from datetime import datetime, timedelta # Get all Form 4 filings for a company in the last 90 days end_date = datetime.today() start_date = end_date - timedelta(days=90) company = Company("NVDA") recent_form4 = company.get_filings( form="4", start_date=start_date.strftime("%Y-%m-%d"), end_date=end_date.strftime("%Y-%m-%d") ) # Analyze all filings transactions_data = [] for filing in recent_form4: try: form4 = filing.obj() net_shares = form4.get_net_shares_traded() transactions_data.append({ 'date': form4.filing_date, 'name': form4.reporting_owner_name, 'relationship': form4.reporting_owner_relationship, 'net_shares': net_shares, 'transaction_type': 'BUY' if net_shares > 0 else 'SELL' if net_shares < 0 else 'NEUTRAL' }) except Exception as e: print(f"Error processing filing {filing.accession_number}: {e}") # Create a DataFrame for analysis df = pd.DataFrame(transactions_data) if not df.empty: # Summarize by person person_summary = df.groupby('name').agg({ 'net_shares': 'sum', 'date': 'count' }).rename(columns={'date': 'num_transactions'}).sort_values('net_shares') print("\nInsider Activity by Person:") print(person_summary) # Summarize by transaction type type_counts = df['transaction_type'].value_counts() print(f"\nTransaction Types: {dict(type_counts)}")` ### Tracking Significant Transactions Identify large or otherwise noteworthy transactions: `def get_significant_transactions(company_ticker, min_value=1000000, days=180): """Find Form 4 transactions above a certain dollar value.""" company = Company(company_ticker) end_date = datetime.today() start_date = end_date - timedelta(days=days) form4_filings = company.get_filings( form="4", start_date=start_date.strftime("%Y-%m-%d"), end_date=end_date.strftime("%Y-%m-%d") ) significant_transactions = [] for filing in form4_filings: try: form4 = filing.obj() for transaction in form4.transactions: if transaction.value and transaction.value >= min_value: significant_transactions.append({ 'date': transaction.transaction_date, 'filing_date': form4.filing_date, 'name': form4.reporting_owner_name, 'relationship': form4.reporting_owner_relationship, 'shares': transaction.shares, 'price': transaction.price_per_share, 'value': transaction.value, 'type': transaction.transaction_code, 'accession': filing.accession_number }) except Exception as e: print(f"Error processing filing {filing.accession_number}: {e}") return pd.DataFrame(significant_transactions).sort_values('value', ascending=False) # Find significant transactions for a company significant_df = get_significant_transactions("AMZN", min_value=5000000) print(f"\nFound {len(significant_df)} significant transactions") if not significant_df.empty: print(significant_df.head())` Advanced Analysis Techniques ---------------------------- ### Correlating with Stock Price Combine insider trading data with stock price data to identify patterns: `import pandas as pd import matplotlib.pyplot as plt import yfinance as yf # You'll need to install this package def analyze_insider_vs_price(ticker, days=180): """Compare insider transactions with stock price movement.""" # Get stock price data end_date = datetime.today() start_date = end_date - timedelta(days=days) stock_data = yf.download(ticker, start=start_date, end=end_date) # Get insider transactions company = Company(ticker) form4_filings = company.get_filings( form="4", start_date=start_date.strftime("%Y-%m-%d"), end_date=end_date.strftime("%Y-%m-%d") ) # Process transactions insider_data = [] for filing in form4_filings: try: form4 = filing.obj() net_shares = form4.get_net_shares_traded() if net_shares != 0: # Only include actual buys or sells insider_data.append({ 'date': pd.to_datetime(form4.filing_date), 'net_shares': net_shares, 'transaction_type': 'BUY' if net_shares > 0 else 'SELL' }) except Exception as e: print(f"Error processing filing: {e}") insider_df = pd.DataFrame(insider_data) # Skip plotting if we don't have both datasets if insider_df.empty or stock_data.empty: print("Insufficient data for analysis") return # Create a plot plt.figure(figsize=(12, 6)) # Plot stock price plt.plot(stock_data.index, stock_data['Close'], label='Stock Price') # Mark insider transactions for _, row in insider_df.iterrows(): color = 'green' if row['transaction_type'] == 'BUY' else 'red' marker = '^' if row['transaction_type'] == 'BUY' else 'v' plt.scatter(row['date'], stock_data.loc[stock_data.index >= row['date']].iloc[0]['Close'], color=color, s=100, marker=marker) plt.title(f'{ticker} Stock Price vs Insider Transactions') plt.legend(['Stock Price', 'Insider Buy', 'Insider Sell']) plt.grid(True) plt.savefig(f'{ticker}_insider_analysis.png') plt.close() return insider_df, stock_data # Run the analysis analyze_insider_vs_price("MSFT")` Best Practices and Tips ----------------------- ### Handling Transaction Complexities Form 4 filings can have complexities to watch out for: 1. **Multiple Transactions**: A single Form 4 can contain multiple transactions 2. **Amended Filings**: Form 4/A filings are amendments to previous filings 3. **Indirect Ownership**: Transactions might involve indirect ownership through trusts or other entities 4. **Derivative Securities**: Some transactions involve options, warrants, or other derivatives Handle these cases with careful code: `def process_form4_safely(filing): try: # Check if this is an amended filing if filing.form_type == "4/A": print(f"This is an amended filing: {filing.accession_number}") form4 = filing.obj() # Handle multiple transactions transaction_count = len(form4.transactions) if transaction_count > 1: print(f"Filing has {transaction_count} transactions") # Check for indirect ownership for transaction in form4.transactions: if transaction.ownership == "I": # Indirect ownership print(f"Indirect ownership transaction found: {transaction.ownership_nature}") # Check for derivative securities if hasattr(form4, 'derivative_transactions') and form4.derivative_transactions: print(f"Filing includes {len(form4.derivative_transactions)} derivative transactions") return form4 except Exception as e: print(f"Error processing Form 4: {e}") return None` ### Performance Considerations When working with large volumes of Form 4 filings: 1. **Use Local Storage**: Store filings locally to avoid repeated downloads 2. **Process in Batches**: Process filings in manageable batches 3. **Filter Early**: Apply filters early in your pipeline to reduce the dataset size `from edgar import enable_local_storage # Enable local storage enable_local_storage("/path/to/storage") # Process filings in batches all_filings = get_filings(form="4", year=2024) batch_size = 100 for i in range(0, len(all_filings), batch_size): batch = all_filings[i:i+batch_size] print(f"Processing batch {i//batch_size + 1} ({len(batch)} filings)") # Process this batch for filing in batch: # Your processing code here pass` Conclusion ---------- Tracking insider trading with Form 4 filings can provide valuable insights into the sentiment of company insiders. edgartools makes it easy to retrieve, parse, and analyze these filings at scale, allowing you to incorporate insider trading data into your investment research or analysis workflows. By understanding the structure of Form 4 filings and leveraging edgartools' data objects, you can efficiently extract meaningful insights about insider activity without manual effort. Whether you're tracking transactions by company executives, monitoring significant purchases or sales, or correlating insider activity with stock price movements, edgartools provides the foundation for comprehensive insider trading analysis. Additional Resources -------------------- * [SEC Form 4 Guide](https://www.sec.gov/files/form4.pdf) * [Insider Trading Legal Framework](https://www.sec.gov/Archives/edgar/data/25743/000138713113000737/ex14_02.htm) * [Form 4 Data Objects API Reference](https://edgartools.readthedocs.io/en/latest/data-objects/) Back to top --- # Private Offerings (Form D) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/formd-data-object-guide/#form-d-parse-private-placement-and-regulation-d-filings) Form D: Parse Private Placement and Regulation D Filings ======================================================== Overview -------- **Form D** is an SEC filing used by companies seeking to raise capital through private placements under Regulation D exemptions. These filings provide critical insights into private fundraising activity, including offering amounts, investor counts, and the parties involved in the capital raise. The `FormD` class in edgartools parses Form D XML filings into structured Python objects, making it easy to extract and display offering data programmatically. Access Pattern -------------- `from edgar import Filing # Get a Form D filing filing = Filing(form="D", ...) # Parse into FormD object form_d = filing.obj()` * * * Core Data Structure ------------------- ### FormD (Top-Level Object) | Property | Type | Description | | --- | --- | --- | | `submission_type` | `str` | Filing type (e.g., "D", "D/A" for amendments) | | `is_live` | `bool` | Whether this is a live filing (vs. test) | | `is_new` | `bool` | Whether this is a new offering (vs. amendment) | | `primary_issuer` | `Issuer` | The company raising capital | | `related_persons` | `List[Person]` | Executives, directors, and promoters involved | | `offering_data` | `OfferingData` | Details about the capital raise | | `signature_block` | `SignatureBlock` | Filing signatures and authorization | * * * Issuer Information ------------------ ### Issuer The `primary_issuer` property contains the company raising capital. | Property | Type | Description | UI Usage | | --- | --- | --- | --- | | `cik` | `str` | SEC Central Index Key | Link to SEC filings | | `entity_name` | `str` | Legal name of company | Primary display name | | `entity_type` | `str` | Legal structure (LLC, Corporation, LP, etc.) | Company type badge | | `primary_address` | `Address` | Business address | Contact/location display | | `phone_number` | `str` | Contact phone | Contact info | | `jurisdiction` | `str` | State/country of incorporation | Jurisdiction badge | | `year_of_incorporation` | `str` | Year company was formed | Age indicator | | `incorporated_within_5_years` | `bool` | Recently formed flag | Startup indicator | | `issuer_previous_names` | `List[str]` | Prior company names | Name history | | `edgar_previous_names` | `List[str]` | Prior names in SEC system | Name history | ### Address | Property | Type | Description | | --- | --- | --- | | `street1` | `str` | Street address line 1 | | `street2` | `str` | Street address line 2 (optional) | | `city` | `str` | City | | `state_or_country` | `str` | State/country code (e.g., "CA", "DE") | | `state_or_country_description` | `str` | Full state/country name | | `zipcode` | `str` | Postal code | | `empty` | `bool` (property) | True if address has no data | * * * Offering Details ---------------- ### OfferingData The `offering_data` property contains the core capital raise information. | Property | Type | Description | UI Usage | | --- | --- | --- | --- | | `industry_group` | `IndustryGroup` | Industry classification | Industry filtering/display | | `revenue_range` | `str` | Issuer's revenue bracket | Size indicator | | `federal_exemptions` | `List[str]` | Reg D exemptions claimed (506(b), 506(c), etc.) | Exemption badges | | `is_new` | `bool` | New offering vs. amendment | Status indicator | | `date_of_first_sale` | `str` | When sales began | Timeline display | | `more_than_one_year` | `bool` | Offering duration flag | Duration indicator | | `is_equity` | `bool` | Equity securities offered | Security type badge | | `is_pooled_investment` | `bool` | Pooled investment fund type | Fund type indicator | | `business_combination_transaction` | `BusinessCombinationTransaction` | M&A related flag | Transaction type | | `minimum_investment` | `str` | Minimum investor commitment | Investment threshold | | `offering_sales_amounts` | `OfferingSalesAmounts` | Capital raise metrics | Key financial metrics | | `investors` | `Investors` | Investor information | Investor stats | | `sales_compensation_recipients` | `List[SalesCompensationRecipient]` | Brokers/finders involved | Sales team display | | `sales_commission_finders_fees` | `SalesCommissionFindersFees` | Commission amounts | Fee breakdown | | `use_of_proceeds` | `UseOfProceeds` | How funds will be used | Proceeds allocation | ### OfferingSalesAmounts **Key metrics for displaying offering progress.** | Property | Type | Description | UI Usage | | --- | --- | --- | --- | | `total_offering_amount` | `object` | Target raise amount | Progress bar max | | `total_amount_sold` | `object` | Amount already raised | Progress bar current | | `total_remaining` | `object` | Amount still to raise | Progress bar remaining | | `clarification_of_response` | `str` | Additional notes | Tooltip/details | ### Investors | Property | Type | Description | UI Usage | | --- | --- | --- | --- | | `has_non_accredited_investors` | `bool` | Non-accredited investors allowed | Investor type badge | | `total_already_invested` | `object` | Number of investors | Investor count display | ### IndustryGroup | Property | Type | Description | | --- | --- | --- | | `industry_group_type` | `str` | Industry category (Real Estate, Technology, etc.) | | `investment_fund_info` | `InvestmentFundInfo` | Fund-specific info if applicable | ### InvestmentFundInfo (for pooled funds) | Property | Type | Description | | --- | --- | --- | | `investment_fund_type` | `str` | Fund type (Hedge Fund, Private Equity, Venture Capital, etc.) | | `is_40_act` | `bool` | Whether fund is registered under Investment Company Act | * * * Sales Compensation ------------------ ### SalesCompensationRecipient **Brokers, finders, and placement agents who receive compensation for the offering.** | Property | Type | Description | UI Usage | | --- | --- | --- | --- | | `name` | `str` | Individual/firm name | Broker name display | | `crd` | `str` | FINRA CRD number | BrokerCheck link | | `associated_bd_name` | `str` | Associated broker-dealer name | BD relationship | | `associated_bd_crd` | `str` | BD's CRD number | BD BrokerCheck link | | `address` | `Address` | Contact address | Location display | | `states_of_solicitation` | `List[str]` | States where they can solicit | Geographic scope | ### SalesCommissionFindersFees | Property | Type | Description | | --- | --- | --- | | `sales_commission` | `object` | Total sales commissions paid | | `finders_fees` | `object` | Total finders fees paid | | `clarification_of_response` | `str` | Additional notes | ### UseOfProceeds | Property | Type | Description | | --- | --- | --- | | `gross_proceeds_used` | `object` | Amount of proceeds already used | | `clarification_of_response` | `str` | Description of use | * * * Related Persons --------------- ### Person **Executives, directors, and promoters associated with the offering.** | Property | Type | Description | | --- | --- | --- | | `first_name` | `str` | Person's first name | | `last_name` | `str` | Person's last name | | `address` | `Address` | Contact address | * * * Signatures ---------- ### SignatureBlock | Property | Type | Description | | --- | --- | --- | | `authorized_representative` | `bool` | Whether signer is authorized representative | | `signatures` | `List[Signature]` | List of signatures on filing | ### Signature | Property | Type | Description | | --- | --- | --- | | `issuer_name` | `str` | Name of issuing entity | | `signature_name` | `str` | Signature as signed | | `name_of_signer` | `str` | Signer's printed name | | `title` | `str` | Signer's title | | `date` | `str` | Date signed | * * * Federal Exemptions Reference ---------------------------- Common exemption codes found in `federal_exemptions`: | Code | Description | | --- | --- | | `06b` | Rule 506(b) - Private placement, no general solicitation | | `06c` | Rule 506(c) - General solicitation allowed, accredited only | | `04` | Rule 504 - Up to $10M in 12 months | | `3C` | Section 3(c) - Investment company exemption | | `3C.1` | Section 3(c)(1) - 100 investor limit | | `3C.7` | Section 3(c)(7) - Qualified purchasers only | * * * Industry Group Types -------------------- Common values for `industry_group_type`: * `Real Estate` * `Banking & Financial Services` * `Technology` * `Health Care` * `Manufacturing` * `Retailing` * `Energy` * `Pooled Investment Fund` * `Other` * * * UI Component Recommendations ---------------------------- ### Summary Card Display the key offering metrics prominently: - Issuer name and entity type - Total offering amount / amount sold (progress indicator) - Number of investors - Minimum investment - Date of first sale - Federal exemptions (as badges) ### Issuer Details Panel * Entity name, type, jurisdiction * Year of incorporation (with "startup" badge if < 5 years) * Address and phone * Previous names (if any) ### Offering Metrics Dashboard * Offering amount vs. sold (bar chart or progress ring) * Investor count * Minimum investment threshold * Security type (equity badge) * Duration indicator ### Related Persons Table * Name column * Address column (expandable) * Useful for due diligence ### Sales Compensation Table * Broker/finder name * CRD number (link to FINRA BrokerCheck) * Associated broker-dealer * States of solicitation (collapsed list with expand) ### Signatures Panel * Signer name and title * Date signed * Authorization status * * * Common Queries and Filters -------------------------- For SAAS features, consider enabling filters by: 1. **Offering Size** - Filter by `total_offering_amount` ranges 2. **Industry** - Filter by `industry_group_type` 3. **Exemption Type** - Filter by `federal_exemptions` (506(b) vs 506(c)) 4. **Investor Type** - Filter by `has_non_accredited_investors` 5. **Entity Type** - Filter by issuer `entity_type` 6. **Jurisdiction** - Filter by issuer `jurisdiction` 7. **Date Range** - Filter by `date_of_first_sale` 8. **New vs Amendment** - Filter by `is_new` * * * Data Quality Notes ------------------ 1. **Monetary amounts** - Stored as strings/objects; may need parsing for numeric operations 2. **Optional fields** - Many fields can be `None`; handle gracefully in UI 3. **States of solicitation** - May contain "All States" as a single value 4. **Previous names** - Filter out literal "None" strings 5. **Date formats** - May vary; normalize for display * * * Example Data Access ------------------- `# Get filing and parse form_d = filing.obj() # Access issuer info print(form_d.primary_issuer.entity_name) print(form_d.primary_issuer.jurisdiction) # Access offering metrics print(form_d.offering_data.offering_sales_amounts.total_offering_amount) print(form_d.offering_data.offering_sales_amounts.total_amount_sold) print(form_d.offering_data.investors.total_already_invested) # List exemptions for exemption in form_d.offering_data.federal_exemptions: print(f"Exemption: {exemption}") # List related persons for person in form_d.related_persons: print(f"{person.first_name} {person.last_name}") # List sales compensation recipients for recipient in form_d.offering_data.sales_compensation_recipients: print(f"{recipient.name} (CRD: {recipient.crd})") print(f" States: {', '.join(recipient.states_of_solicitation)}")` Back to top --- # Track Insider Trading: Analyze SEC Form 4 Buy and Sell Transactions - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/track-form4/#track-insider-trading-analyze-sec-form-4-buy-and-sell-transactions) Track Insider Trading: Analyze SEC Form 4 Buy and Sell Transactions =================================================================== Introduction ------------ Form 4 filings provide valuable insights into insider trading activity. When corporate insiders (directors, officers, or beneficial owners of more than 10% of a company's stock) buy or sell shares of their company, they must report these transactions to the SEC via Form 4 filings. These filings can reveal important signals about insiders' confidence in their company's future. edgartools makes it easy to retrieve, parse, and analyze Form 4 filings programmatically, allowing you to track insider trading patterns without manual effort. Understanding Form 4 Filings ---------------------------- Before diving into code, it's important to understand what Form 4 filings contain: * **Reporting Person Information**: Name, relationship to the company (e.g., CEO, Director) * **Transaction Details**: Date, type of security, number of shares, price per share * **Transaction Codes**: Codes that indicate the nature of the transaction (e.g., P for purchase, S for sale) * **Ownership Information**: Direct or indirect ownership, total shares held after transaction Retrieving Form 4 Filings ------------------------- ### By Company To retrieve Form 4 filings for a specific company: `from edgar import Company, get_filings # Using Company object company = Company("AAPL") form4_filings = company.get_filings(form="4") # Or using global get_filings form4_filings = get_filings(form="4", ticker="AAPL") # View the most recent filings recent_filings = form4_filings.head(5) for filing in recent_filings: print(f"Date: {filing.filing_date}, Person: {filing.reporting_owner_name}")` ### By Date Range To find Form 4 filings within a specific date range: `# Get Form 4 filings from Jan 1, 2024 to present form4_filings = get_filings( form="4", ticker="MSFT", start_date="2024-01-01", end_date="2024-07-01" ) print(f"Found {len(form4_filings)} Form 4 filings")` ### By Reporting Person To focus on a specific insider's activity: `form4_filings = get_filings(form="4", ticker="TSLA") # Filter by reporting person's name musk_filings = form4_filings.filter(reporting_owner_name="Musk Elon") print(f"Found {len(musk_filings)} Form 4 filings by Elon Musk")` Working with Form 4 Data Objects -------------------------------- edgartools provides a specialized `Form4` data object that makes it easy to access structured data from these filings: `# Get a specific Form 4 filing filing = form4_filings.latest() # Convert to Form4 data object form4 = filing.obj() # Access basic metadata print(f"Filing date: {form4.filing_date}") print(f"Reporting owner: {form4.reporting_owner_name}") print(f"Relationship: {form4.reporting_owner_relationship}") print(f"Company: {form4.issuer_name} ({form4.issuer_ticker})")` ### Accessing Transaction Details Form 4 filings can contain multiple transactions. Access them through the `transactions` property: `# Examine all transactions in the filing for i, transaction in enumerate(form4.transactions): print(f"\nTransaction {i+1}:") print(f"Date: {transaction.transaction_date}") print(f"Type: {transaction.transaction_code} ({transaction.get_transaction_code_description()})") print(f"Shares: {transaction.shares}") print(f"Price: ${transaction.price_per_share:.2f}") print(f"Value: ${transaction.value:.2f}") print(f"Direct/Indirect: {transaction.ownership}") print(f"Shares owned after: {transaction.shares_owned_following_transaction}")` ### Understanding Transaction Codes Form 4 transactions use codes to indicate different types of transactions: `# Common transaction codes and their meanings transaction_codes = { 'P': 'Open market or private purchase of securities', 'S': 'Open market or private sale of securities', 'A': 'Grant, award, or other acquisition', 'D': 'Disposition to the issuer (e.g., forfeiture, cancellation)', 'M': 'Exercise or conversion of derivative security', 'G': 'Gift', 'V': 'Voluntary transaction with issuer' } # Check what type of transaction this is for transaction in form4.transactions: code = transaction.transaction_code description = transaction_codes.get(code, "Other transaction type") print(f"Transaction code {code}: {description}") print(f"Shares: {transaction.shares}")` Analyzing Insider Transactions ------------------------------ ### Calculating Net Shares Traded Calculate whether an insider is buying or selling on net: `# Calculate net shares traded in a filing net_shares = form4.get_net_shares_traded() if net_shares > 0: print(f"Insider BOUGHT a net {net_shares:,} shares") elif net_shares < 0: print(f"Insider SOLD a net {abs(net_shares):,} shares") else: print("Insider had no net change in position")` ### Aggregating Transactions by Company Track recent insider activity for a company: `import pandas as pd from datetime import datetime, timedelta # Get all Form 4 filings for a company in the last 90 days end_date = datetime.today() start_date = end_date - timedelta(days=90) company = Company("NVDA") recent_form4 = company.get_filings( form="4", start_date=start_date.strftime("%Y-%m-%d"), end_date=end_date.strftime("%Y-%m-%d") ) # Analyze all filings transactions_data = [] for filing in recent_form4: try: form4 = filing.obj() net_shares = form4.get_net_shares_traded() transactions_data.append({ 'date': form4.filing_date, 'name': form4.reporting_owner_name, 'relationship': form4.reporting_owner_relationship, 'net_shares': net_shares, 'transaction_type': 'BUY' if net_shares > 0 else 'SELL' if net_shares < 0 else 'NEUTRAL' }) except Exception as e: print(f"Error processing filing {filing.accession_number}: {e}") # Create a DataFrame for analysis df = pd.DataFrame(transactions_data) if not df.empty: # Summarize by person person_summary = df.groupby('name').agg({ 'net_shares': 'sum', 'date': 'count' }).rename(columns={'date': 'num_transactions'}).sort_values('net_shares') print("\nInsider Activity by Person:") print(person_summary) # Summarize by transaction type type_counts = df['transaction_type'].value_counts() print(f"\nTransaction Types: {dict(type_counts)}")` ### Tracking Significant Transactions Identify large or otherwise noteworthy transactions: `def get_significant_transactions(company_ticker, min_value=1000000, days=180): """Find Form 4 transactions above a certain dollar value.""" company = Company(company_ticker) end_date = datetime.today() start_date = end_date - timedelta(days=days) form4_filings = company.get_filings( form="4", start_date=start_date.strftime("%Y-%m-%d"), end_date=end_date.strftime("%Y-%m-%d") ) significant_transactions = [] for filing in form4_filings: try: form4 = filing.obj() for transaction in form4.transactions: if transaction.value and transaction.value >= min_value: significant_transactions.append({ 'date': transaction.transaction_date, 'filing_date': form4.filing_date, 'name': form4.reporting_owner_name, 'relationship': form4.reporting_owner_relationship, 'shares': transaction.shares, 'price': transaction.price_per_share, 'value': transaction.value, 'type': transaction.transaction_code, 'accession': filing.accession_number }) except Exception as e: print(f"Error processing filing {filing.accession_number}: {e}") return pd.DataFrame(significant_transactions).sort_values('value', ascending=False) # Find significant transactions for a company significant_df = get_significant_transactions("AMZN", min_value=5000000) print(f"\nFound {len(significant_df)} significant transactions") if not significant_df.empty: print(significant_df.head())` Advanced Analysis Techniques ---------------------------- ### Correlating with Stock Price Combine insider trading data with stock price data to identify patterns: `import pandas as pd import matplotlib.pyplot as plt import yfinance as yf # You'll need to install this package def analyze_insider_vs_price(ticker, days=180): """Compare insider transactions with stock price movement.""" # Get stock price data end_date = datetime.today() start_date = end_date - timedelta(days=days) stock_data = yf.download(ticker, start=start_date, end=end_date) # Get insider transactions company = Company(ticker) form4_filings = company.get_filings( form="4", start_date=start_date.strftime("%Y-%m-%d"), end_date=end_date.strftime("%Y-%m-%d") ) # Process transactions insider_data = [] for filing in form4_filings: try: form4 = filing.obj() net_shares = form4.get_net_shares_traded() if net_shares != 0: # Only include actual buys or sells insider_data.append({ 'date': pd.to_datetime(form4.filing_date), 'net_shares': net_shares, 'transaction_type': 'BUY' if net_shares > 0 else 'SELL' }) except Exception as e: print(f"Error processing filing: {e}") insider_df = pd.DataFrame(insider_data) # Skip plotting if we don't have both datasets if insider_df.empty or stock_data.empty: print("Insufficient data for analysis") return # Create a plot plt.figure(figsize=(12, 6)) # Plot stock price plt.plot(stock_data.index, stock_data['Close'], label='Stock Price') # Mark insider transactions for _, row in insider_df.iterrows(): color = 'green' if row['transaction_type'] == 'BUY' else 'red' marker = '^' if row['transaction_type'] == 'BUY' else 'v' plt.scatter(row['date'], stock_data.loc[stock_data.index >= row['date']].iloc[0]['Close'], color=color, s=100, marker=marker) plt.title(f'{ticker} Stock Price vs Insider Transactions') plt.legend(['Stock Price', 'Insider Buy', 'Insider Sell']) plt.grid(True) plt.savefig(f'{ticker}_insider_analysis.png') plt.close() return insider_df, stock_data # Run the analysis analyze_insider_vs_price("MSFT")` Best Practices and Tips ----------------------- ### Handling Transaction Complexities Form 4 filings can have complexities to watch out for: 1. **Multiple Transactions**: A single Form 4 can contain multiple transactions 2. **Amended Filings**: Form 4/A filings are amendments to previous filings 3. **Indirect Ownership**: Transactions might involve indirect ownership through trusts or other entities 4. **Derivative Securities**: Some transactions involve options, warrants, or other derivatives Handle these cases with careful code: `def process_form4_safely(filing): try: # Check if this is an amended filing if filing.form_type == "4/A": print(f"This is an amended filing: {filing.accession_number}") form4 = filing.obj() # Handle multiple transactions transaction_count = len(form4.transactions) if transaction_count > 1: print(f"Filing has {transaction_count} transactions") # Check for indirect ownership for transaction in form4.transactions: if transaction.ownership == "I": # Indirect ownership print(f"Indirect ownership transaction found: {transaction.ownership_nature}") # Check for derivative securities if hasattr(form4, 'derivative_transactions') and form4.derivative_transactions: print(f"Filing includes {len(form4.derivative_transactions)} derivative transactions") return form4 except Exception as e: print(f"Error processing Form 4: {e}") return None` ### Performance Considerations When working with large volumes of Form 4 filings: 1. **Use Local Storage**: Store filings locally to avoid repeated downloads 2. **Process in Batches**: Process filings in manageable batches 3. **Filter Early**: Apply filters early in your pipeline to reduce the dataset size `from edgar import enable_local_storage # Enable local storage enable_local_storage("/path/to/storage") # Process filings in batches all_filings = get_filings(form="4", year=2024) batch_size = 100 for i in range(0, len(all_filings), batch_size): batch = all_filings[i:i+batch_size] print(f"Processing batch {i//batch_size + 1} ({len(batch)} filings)") # Process this batch for filing in batch: # Your processing code here pass` Conclusion ---------- Tracking insider trading with Form 4 filings can provide valuable insights into the sentiment of company insiders. edgartools makes it easy to retrieve, parse, and analyze these filings at scale, allowing you to incorporate insider trading data into your investment research or analysis workflows. By understanding the structure of Form 4 filings and leveraging edgartools' data objects, you can efficiently extract meaningful insights about insider activity without manual effort. Whether you're tracking transactions by company executives, monitoring significant purchases or sales, or correlating insider activity with stock price movements, edgartools provides the foundation for comprehensive insider trading analysis. Additional Resources -------------------- * [SEC Form 4 Guide](https://www.sec.gov/files/form4.pdf) * [Insider Trading Legal Framework](https://www.sec.gov/Archives/edgar/data/25743/000138713113000737/ex14_02.htm) * [Form 4 Data Objects API Reference](https://edgartools.readthedocs.io/en/stable/data-objects/) Back to top --- # Private Offerings (Form D) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/formd-data-object-guide/#form-d-parse-private-placement-and-regulation-d-filings) Form D: Parse Private Placement and Regulation D Filings ======================================================== Overview -------- **Form D** is an SEC filing used by companies seeking to raise capital through private placements under Regulation D exemptions. These filings provide critical insights into private fundraising activity, including offering amounts, investor counts, and the parties involved in the capital raise. The `FormD` class in edgartools parses Form D XML filings into structured Python objects, making it easy to extract and display offering data programmatically. Access Pattern -------------- `from edgar import Filing # Get a Form D filing filing = Filing(form="D", ...) # Parse into FormD object form_d = filing.obj()` * * * Core Data Structure ------------------- ### FormD (Top-Level Object) | Property | Type | Description | | --- | --- | --- | | `submission_type` | `str` | Filing type (e.g., "D", "D/A" for amendments) | | `is_live` | `bool` | Whether this is a live filing (vs. test) | | `is_new` | `bool` | Whether this is a new offering (vs. amendment) | | `primary_issuer` | `Issuer` | The company raising capital | | `related_persons` | `List[Person]` | Executives, directors, and promoters involved | | `offering_data` | `OfferingData` | Details about the capital raise | | `signature_block` | `SignatureBlock` | Filing signatures and authorization | * * * Issuer Information ------------------ ### Issuer The `primary_issuer` property contains the company raising capital. | Property | Type | Description | UI Usage | | --- | --- | --- | --- | | `cik` | `str` | SEC Central Index Key | Link to SEC filings | | `entity_name` | `str` | Legal name of company | Primary display name | | `entity_type` | `str` | Legal structure (LLC, Corporation, LP, etc.) | Company type badge | | `primary_address` | `Address` | Business address | Contact/location display | | `phone_number` | `str` | Contact phone | Contact info | | `jurisdiction` | `str` | State/country of incorporation | Jurisdiction badge | | `year_of_incorporation` | `str` | Year company was formed | Age indicator | | `incorporated_within_5_years` | `bool` | Recently formed flag | Startup indicator | | `issuer_previous_names` | `List[str]` | Prior company names | Name history | | `edgar_previous_names` | `List[str]` | Prior names in SEC system | Name history | ### Address | Property | Type | Description | | --- | --- | --- | | `street1` | `str` | Street address line 1 | | `street2` | `str` | Street address line 2 (optional) | | `city` | `str` | City | | `state_or_country` | `str` | State/country code (e.g., "CA", "DE") | | `state_or_country_description` | `str` | Full state/country name | | `zipcode` | `str` | Postal code | | `empty` | `bool` (property) | True if address has no data | * * * Offering Details ---------------- ### OfferingData The `offering_data` property contains the core capital raise information. | Property | Type | Description | UI Usage | | --- | --- | --- | --- | | `industry_group` | `IndustryGroup` | Industry classification | Industry filtering/display | | `revenue_range` | `str` | Issuer's revenue bracket | Size indicator | | `federal_exemptions` | `List[str]` | Reg D exemptions claimed (506(b), 506(c), etc.) | Exemption badges | | `is_new` | `bool` | New offering vs. amendment | Status indicator | | `date_of_first_sale` | `str` | When sales began | Timeline display | | `more_than_one_year` | `bool` | Offering duration flag | Duration indicator | | `is_equity` | `bool` | Equity securities offered | Security type badge | | `is_pooled_investment` | `bool` | Pooled investment fund type | Fund type indicator | | `business_combination_transaction` | `BusinessCombinationTransaction` | M&A related flag | Transaction type | | `minimum_investment` | `str` | Minimum investor commitment | Investment threshold | | `offering_sales_amounts` | `OfferingSalesAmounts` | Capital raise metrics | Key financial metrics | | `investors` | `Investors` | Investor information | Investor stats | | `sales_compensation_recipients` | `List[SalesCompensationRecipient]` | Brokers/finders involved | Sales team display | | `sales_commission_finders_fees` | `SalesCommissionFindersFees` | Commission amounts | Fee breakdown | | `use_of_proceeds` | `UseOfProceeds` | How funds will be used | Proceeds allocation | ### OfferingSalesAmounts **Key metrics for displaying offering progress.** | Property | Type | Description | UI Usage | | --- | --- | --- | --- | | `total_offering_amount` | `object` | Target raise amount | Progress bar max | | `total_amount_sold` | `object` | Amount already raised | Progress bar current | | `total_remaining` | `object` | Amount still to raise | Progress bar remaining | | `clarification_of_response` | `str` | Additional notes | Tooltip/details | ### Investors | Property | Type | Description | UI Usage | | --- | --- | --- | --- | | `has_non_accredited_investors` | `bool` | Non-accredited investors allowed | Investor type badge | | `total_already_invested` | `object` | Number of investors | Investor count display | ### IndustryGroup | Property | Type | Description | | --- | --- | --- | | `industry_group_type` | `str` | Industry category (Real Estate, Technology, etc.) | | `investment_fund_info` | `InvestmentFundInfo` | Fund-specific info if applicable | ### InvestmentFundInfo (for pooled funds) | Property | Type | Description | | --- | --- | --- | | `investment_fund_type` | `str` | Fund type (Hedge Fund, Private Equity, Venture Capital, etc.) | | `is_40_act` | `bool` | Whether fund is registered under Investment Company Act | * * * Sales Compensation ------------------ ### SalesCompensationRecipient **Brokers, finders, and placement agents who receive compensation for the offering.** | Property | Type | Description | UI Usage | | --- | --- | --- | --- | | `name` | `str` | Individual/firm name | Broker name display | | `crd` | `str` | FINRA CRD number | BrokerCheck link | | `associated_bd_name` | `str` | Associated broker-dealer name | BD relationship | | `associated_bd_crd` | `str` | BD's CRD number | BD BrokerCheck link | | `address` | `Address` | Contact address | Location display | | `states_of_solicitation` | `List[str]` | States where they can solicit | Geographic scope | ### SalesCommissionFindersFees | Property | Type | Description | | --- | --- | --- | | `sales_commission` | `object` | Total sales commissions paid | | `finders_fees` | `object` | Total finders fees paid | | `clarification_of_response` | `str` | Additional notes | ### UseOfProceeds | Property | Type | Description | | --- | --- | --- | | `gross_proceeds_used` | `object` | Amount of proceeds already used | | `clarification_of_response` | `str` | Description of use | * * * Related Persons --------------- ### Person **Executives, directors, and promoters associated with the offering.** | Property | Type | Description | | --- | --- | --- | | `first_name` | `str` | Person's first name | | `last_name` | `str` | Person's last name | | `address` | `Address` | Contact address | * * * Signatures ---------- ### SignatureBlock | Property | Type | Description | | --- | --- | --- | | `authorized_representative` | `bool` | Whether signer is authorized representative | | `signatures` | `List[Signature]` | List of signatures on filing | ### Signature | Property | Type | Description | | --- | --- | --- | | `issuer_name` | `str` | Name of issuing entity | | `signature_name` | `str` | Signature as signed | | `name_of_signer` | `str` | Signer's printed name | | `title` | `str` | Signer's title | | `date` | `str` | Date signed | * * * Federal Exemptions Reference ---------------------------- Common exemption codes found in `federal_exemptions`: | Code | Description | | --- | --- | | `06b` | Rule 506(b) - Private placement, no general solicitation | | `06c` | Rule 506(c) - General solicitation allowed, accredited only | | `04` | Rule 504 - Up to $10M in 12 months | | `3C` | Section 3(c) - Investment company exemption | | `3C.1` | Section 3(c)(1) - 100 investor limit | | `3C.7` | Section 3(c)(7) - Qualified purchasers only | * * * Industry Group Types -------------------- Common values for `industry_group_type`: * `Real Estate` * `Banking & Financial Services` * `Technology` * `Health Care` * `Manufacturing` * `Retailing` * `Energy` * `Pooled Investment Fund` * `Other` * * * UI Component Recommendations ---------------------------- ### Summary Card Display the key offering metrics prominently: - Issuer name and entity type - Total offering amount / amount sold (progress indicator) - Number of investors - Minimum investment - Date of first sale - Federal exemptions (as badges) ### Issuer Details Panel * Entity name, type, jurisdiction * Year of incorporation (with "startup" badge if < 5 years) * Address and phone * Previous names (if any) ### Offering Metrics Dashboard * Offering amount vs. sold (bar chart or progress ring) * Investor count * Minimum investment threshold * Security type (equity badge) * Duration indicator ### Related Persons Table * Name column * Address column (expandable) * Useful for due diligence ### Sales Compensation Table * Broker/finder name * CRD number (link to FINRA BrokerCheck) * Associated broker-dealer * States of solicitation (collapsed list with expand) ### Signatures Panel * Signer name and title * Date signed * Authorization status * * * Common Queries and Filters -------------------------- For SAAS features, consider enabling filters by: 1. **Offering Size** - Filter by `total_offering_amount` ranges 2. **Industry** - Filter by `industry_group_type` 3. **Exemption Type** - Filter by `federal_exemptions` (506(b) vs 506(c)) 4. **Investor Type** - Filter by `has_non_accredited_investors` 5. **Entity Type** - Filter by issuer `entity_type` 6. **Jurisdiction** - Filter by issuer `jurisdiction` 7. **Date Range** - Filter by `date_of_first_sale` 8. **New vs Amendment** - Filter by `is_new` * * * Data Quality Notes ------------------ 1. **Monetary amounts** - Stored as strings/objects; may need parsing for numeric operations 2. **Optional fields** - Many fields can be `None`; handle gracefully in UI 3. **States of solicitation** - May contain "All States" as a single value 4. **Previous names** - Filter out literal "None" strings 5. **Date formats** - May vary; normalize for display * * * Example Data Access ------------------- `# Get filing and parse form_d = filing.obj() # Access issuer info print(form_d.primary_issuer.entity_name) print(form_d.primary_issuer.jurisdiction) # Access offering metrics print(form_d.offering_data.offering_sales_amounts.total_offering_amount) print(form_d.offering_data.offering_sales_amounts.total_amount_sold) print(form_d.offering_data.investors.total_already_invested) # List exemptions for exemption in form_d.offering_data.federal_exemptions: print(f"Exemption: {exemption}") # List related persons for person in form_d.related_persons: print(f"{person.first_name} {person.last_name}") # List sales compensation recipients for recipient in form_d.offering_data.sales_compensation_recipients: print(f"{recipient.name} (CRD: {recipient.crd})") print(f" States: {', '.join(recipient.states_of_solicitation)}")` Back to top --- # Prospectus Supplements (424B) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/prospectus424b-data-object-guide/#prospectus-supplements-424b-parse-offering-terms-from-sec-filings) Prospectus Supplements (424B): Parse Offering Terms from SEC Filings ==================================================================== Overview -------- **424B filings** are prospectus supplements that companies file when they sell securities off a shelf registration (S-3/F-3). They contain the final deal terms: price, shares, proceeds, underwriting fees, and dilution impact. EdgarTools parses all 424B variants into a `Prospectus424B` object, with a `Deal` property that normalizes everything into clean numeric values. | Form | Typical Use | | --- | --- | | 424B2 | Structured notes, debt (large banks) | | 424B3 | Resale prospectuses (PIPE resales) | | 424B4 | Final priced prospectuses (IPOs) | | 424B5 | Shelf takedowns (ATM, firm commitment, PIPE) | Quick Start ----------- `from edgar import Company company = Company("ALZN") filing = company.get_filings(form="424B5")[0] prospectus = filing.obj() # Prospectus424B deal = prospectus.deal # Deal: normalized summary deal.price # 2.48 deal.shares # 1_500_000 deal.gross_proceeds # 3_720_000.0 deal.lead_bookrunner # "H.C. Wainwright & Co."` The Deal Object --------------- Access via `prospectus.deal`. Always returns a `Deal` object (never `None`). Individual properties return `None` when data is unavailable. ### Core Deal Terms | Property | Type | Description | | --- | --- | --- | | `price` | `float \\| None` | Per-unit offering price | | `shares` | `int \\| None` | Number of shares offered | | `gross_proceeds` | `float \\| None` | Total offering amount (before fees) | | `net_proceeds` | `float \\| None` | Proceeds after underwriting fees | | `security_type` | `str \\| None` | Security description ("Common Stock", "Senior Notes") | | `offering_type` | `OfferingType` | Enum: `FIRM_COMMITMENT`, `ATM`, `BEST_EFFORTS`, etc. | | `is_atm` | `bool` | Whether this is an at-the-market offering | ### Underwriting Economics | Property | Type | Description | | --- | --- | --- | | `fee_per_share` | `float \\| None` | Per-unit underwriting discount | | `total_fees` | `float \\| None` | Total underwriting fees | | `discount_rate` | `float \\| None` | Fee as fraction of price (0.05 = 5%) | | `fee_type` | `str \\| None` | `"underwriting_discount"` or `"placement_agent_fees"` | | `lead_bookrunner` | `str \\| None` | Lead underwriter or placement agent | | `underwriter_count` | `int` | Number of underwriters in syndicate | ### Dilution (Equity Offerings Only) | Property | Type | Description | | --- | --- | --- | | `dilution_per_share` | `float \\| None` | Dilution to new investors | | `dilution_pct` | `float \\| None` | Dilution as percentage | | `shares_before` | `int \\| None` | Shares outstanding before offering | | `shares_after` | `int \\| None` | Shares outstanding after offering | | `ntbv_before` | `float \\| None` | Net tangible book value per share before | | `ntbv_after` | `float \\| None` | Net tangible book value per share after | ### Serialization `deal.to_dict() # Flat dict of all non-None values (good for DataFrames) deal.to_context() # Markdown-KV text for LLM prompts` Offering Classification ----------------------- The `offering_type` property classifies the deal: | Value | Description | Price/Shares Available? | | --- | --- | --- | | `FIRM_COMMITMENT` | Bank buys all shares, resells | Yes | | `ATM` | At-the-market (sold gradually) | Usually no (market price) | | `BEST_EFFORTS` | Agent sells on best-efforts basis | Yes | | `PIPE_RESALE` | Resale of privately placed shares | Varies | | `STRUCTURED_NOTE` | Bank-issued structured product | Different meaning | | `DEBT_OFFERING` | Corporate bonds / notes | Usually percentage | `if deal.is_atm: # Price and shares are typically None for ATM offerings print(f"ATM program: up to ${deal.gross_proceeds:,.0f}") else: print(f"{deal.shares:,} shares @ ${deal.price:.2f}")` Prospectus Sub-Objects ---------------------- The `Prospectus424B` exposes the raw extracted data that the Deal synthesizes: `prospectus.cover_page # CoverPageData: company, registration, flags prospectus.pricing # PricingData: per-unit and total columns prospectus.underwriting # UnderwritingInfo: syndicate, fee type prospectus.offering_terms # OfferingTerms: shares, warrants, use of proceeds prospectus.selling_stockholders # SellingStockholdersData: PIPE resale tables prospectus.dilution # DilutionData: NTBV impact table prospectus.capitalization # CapitalizationData: actual vs. as-adjusted prospectus.structured_note_terms # StructuredNoteTerms: CUSIP, maturity (424B2) prospectus.filing_fees # FilingFeesData: from XBRL exhibit` Selling Stockholders (PIPE Resale Filings) ------------------------------------------ For PIPE resale prospectuses (typically 424B3), the selling stockholders table lists investors reselling privately placed shares: `ss = prospectus.selling_stockholders # SellingStockholdersData or None if ss: ss.count # Number of selling stockholders for entry in ss.stockholders: entry.name # "Lincoln Park Capital Fund, LLC" entry.shares # 1500000 (parsed int, None on failure) entry.shares_before # 2000000 entry.shares_after # 500000 entry.pct_before # 9.5 (parsed float) entry.pct_after # 2.8 entry.warrants # 750000 (warrants/convertibles, if present)` Raw string values are always preserved (`shares_offered`, `shares_before_offering`, etc.). The numeric properties (`shares`, `shares_before`, etc.) parse them to `int`/`float`, returning `None` on failure. ### DataFrame Output `df = ss.to_dataframe() # Returns DataFrame with numeric columns: # name | shares_before | pct_before | shares_offered | shares_after | pct_after | warrants` ### Offering Type Check `if prospectus.offering_type.has_selling_stockholders: # This is a PIPE_RESALE or BASE_PROSPECTUS_UPDATE ss = prospectus.selling_stockholders` Shelf Lifecycle --------------- Track where a prospectus sits in its shelf registration lifecycle: `lc = prospectus.lifecycle # ShelfLifecycle lc.takedown_number # 3 (this is the 3rd offering) lc.total_takedowns # 5 lc.shelf_expires # date(2027, 8, 2) lc.avg_days_between_takedowns # 180.0 lc.shelf_registration # Filing object for the S-3` Working with Multiple Offerings ------------------------------- Build a DataFrame of a company's offering history: `import pandas as pd from edgar import Company company = Company("ALZN") filings = company.get_filings(form="424B5") rows = [] for filing in filings: prospectus = filing.obj() d = prospectus.deal.to_dict() d['filing_date'] = str(filing.filing_date) rows.append(d) df = pd.DataFrame(rows)` Rich Display ------------ Both `Prospectus424B` and `Deal` render as Rich panels in terminals and notebooks: `prospectus # Shows cover page, pricing table, underwriting prospectus.deal # Compact deal summary panel` Back to top --- # Query XBRL Facts - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/xbrl-querying/#query-xbrl-financial-facts-from-sec-filings) Query XBRL Financial Facts from SEC Filings =========================================== You can query the facts inside an XBRL instance using the XBRL query API. This allows you to get access to specific financial data, filter results and perform analysis on the financial facts contained within a single XBRL filing. Overview -------- XBRL query functionality is built around two main classes: - `FactsView` - Provides access to raw XBRL facts from a single filing - `FactQuery` - Enables complex filtering and analysis of those facts Basic Usage ----------- ### Accessing Facts `from edgar import * from edgar.xbrl import XBRL # Get an XBRL filing company = Company("AAPL") filing = company.latest("10-K") xb = filing.xbrl()` ### Access the facts view The `FactsView` provides direct access to the facts in the XBRL instance: `facts = xb.facts print(f"Total facts: {len(facts)}")` Querying Facts -------------- To query facts use the `query()` method on the `XBRL` instance and one of the `by_` functions e.g. `by_text()`. This returns a `FactQuery` object that allows you to filter and manipulate the facts. `# Start a query results = (xb.query() .by_concept("us-gaap:PaymentsToAcquireAvailableForSaleSecuritiesDebt") )` ![Query AAPL Debt](https://edgartools.readthedocs.io/en/latest/images/query-aapl-debt.png) The result is an `edgar.xbrl.facts.FactQuery` object that contains the filtered facts. You can see from the rich display the available columns of whichg a few are selected by default. You can also convert the results to a DataFrame for easier manipulation including selecting which columns you want to view: `df = results.to_dataframe('concept', 'label', 'value', 'period_end')` Filtering Facts --------------- ### By Concept You can filter facts by their concept names, which are unique identifiers for financial data items in XBRL. `# Find revenue-related facts revenue_query = xb.query().by_concept("us-gaap:Revenues")` The namespace e.g. `us-gaap:` is optional, so you can use just the concept name like `Revenues`. Querying by concept does a partial regex match on the concept name `by_concept('RevenueFrom')` matches `us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax` and `us-gaap:RevenueFromContractWithCustomerTextBlock` Use `exact=True` to match the full concept name exactly. ### By Label You can filter facts by their labels, which are human-readable names associated with the concepts. `# Search by label text revenue_query = xb.query().by_label("Revenue")` To specify exact matches or partial matches, use the `exact` parameter: `sales_query = xb.query().by_label("Revenue", exact=False)` ### By Value `# Facts with values above $1 billion large_values = xb.query().by_value(lambda x: x > 1_000_000_000) # Facts within a range range_query = xb.query().by_value(lambda x: 100_000 <= x <= 1_000_000)` ### By Statement Type `# Facts from specific statements income_facts = xb.query().by_statement_type("IncomeStatement") balance_facts = xb.query().by_statement_type("BalanceSheet")` Method Chaining --------------- Combine multiple filters using method chaining: `# Complex query with multiple filters complex_query = (xbrl.query() .by_statement("IncomeStatement") .by_label("Revenue") .by_value(lambda x: x > 1_000_000) .sort_by('value', ascending=False) .limit(10)) results = complex_query.execute()` Data Transformations -------------------- ### Sorting `# Sort by value (descending) sorted_query = xbrl.query().sort_by('value', ascending=False) # Sort by concept name concept_sorted = xbrl.query().sort_by('concept')` ### Limiting Results `# Get top 10 results top_10 = xbrl.query().limit(10) # Pagination page_1 = xbrl.query().limit(20) page_2 = xbrl.query().offset(20).limit(20)` Working with Results -------------------- ### DataFrame Output `# Get specific columns df = query.to_dataframe('concept', 'label', 'value', 'period_end') # All available columns full_df = query.to_dataframe() # Column information print("Available columns:", df.columns.tolist())` ### Fact Structure Each fact contains the following key information: `fact = results[0] print(f"Concept: {fact['concept']}") print(f"Label: {fact['label']}") print(f"Value: {fact['value']}") print(f"Period: {fact['period_end']}") print(f"Units: {fact['units']}") print(f"Decimals: {fact['decimals']}")` Advanced Filtering ------------------ ### Dimensions `# Facts with specific dimensions dimensional_query = xbrl.query().by_dimension("ProductOrServiceAxis", "ProductMember") # Facts with any value for a dimension any_product_dim = xbrl.query().by_dimension("ProductOrServiceAxis") # Facts with NO dimensions (undimensioned facts) undimensioned_facts = xbrl.query().by_dimension(None) # Multiple dimensions multi_dim = xbrl.query().by_dimensions({ "ProductOrServiceAxis": "ProductMember", "GeographyAxis": "USMember" })` Performance Tips ---------------- 1. **Use specific filters**: Filter early to reduce data processing 2. **Limit results**: Use `.limit()` for large datasets 3. **Cache queries**: Store frequently used queries 4. **Select columns**: Use `to_dataframe()` with specific columns `# Efficient query pattern efficient_query = (xb.query() .by_statement("IncomeStatement") # Filter first .by_value(lambda x: x > 0) # Remove zeros .limit(100) # Limit results .to_dataframe('concept', 'value')) # Select columns` Examples -------- ### Finding Revenue Information `# All revenue-related facts revenue_facts = (xb.query() .by_label("revenue", exact=False) .sort_by('value', ascending=False) .execute()) for fact in revenue_facts: print(f"{fact['label']}: ${fact['value']:,}")` ### Comparing Quarterly Data `# Get quarterly revenue data quarterly_revenue = (xb.query() .by_concept("us-gaap:Revenues") .by_period_type("duration") .sort_by('period_end') .to_dataframe('period_end', 'value')) print(quarterly_revenue)` ### Balance Sheet Analysis `# Major balance sheet items balance_items = (xb.query() .by_statement("BalanceSheet") .by_value(lambda x: x > 1_000_000_000) # > $1B .sort_by('value', ascending=False) .to_dataframe('label', 'value')) print("Major Balance Sheet Items (> $1B):") print(balance_items)` This query system provides a flexible and powerful way to explore XBRL data, enabling detailed financial analysis and data extraction from individual filings. Back to top --- # Query XBRL Facts - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/xbrl-querying/#query-xbrl-financial-facts-from-sec-filings) Query XBRL Financial Facts from SEC Filings =========================================== You can query the facts inside an XBRL instance using the XBRL query API. This allows you to get access to specific financial data, filter results and perform analysis on the financial facts contained within a single XBRL filing. Overview -------- XBRL query functionality is built around two main classes: - `FactsView` - Provides access to raw XBRL facts from a single filing - `FactQuery` - Enables complex filtering and analysis of those facts Basic Usage ----------- ### Accessing Facts `from edgar import * from edgar.xbrl import XBRL # Get an XBRL filing company = Company("AAPL") filing = company.latest("10-K") xb = filing.xbrl()` ### Access the facts view The `FactsView` provides direct access to the facts in the XBRL instance: `facts = xb.facts print(f"Total facts: {len(facts)}")` Querying Facts -------------- To query facts use the `query()` method on the `XBRL` instance and one of the `by_` functions e.g. `by_text()`. This returns a `FactQuery` object that allows you to filter and manipulate the facts. `# Start a query results = (xb.query() .by_concept("us-gaap:PaymentsToAcquireAvailableForSaleSecuritiesDebt") )` ![Query AAPL Debt](https://edgartools.readthedocs.io/en/stable/images/query-aapl-debt.png) The result is an `edgar.xbrl.facts.FactQuery` object that contains the filtered facts. You can see from the rich display the available columns of whichg a few are selected by default. You can also convert the results to a DataFrame for easier manipulation including selecting which columns you want to view: `df = results.to_dataframe('concept', 'label', 'value', 'period_end')` Filtering Facts --------------- ### By Concept You can filter facts by their concept names, which are unique identifiers for financial data items in XBRL. `# Find revenue-related facts revenue_query = xb.query().by_concept("us-gaap:Revenues")` The namespace e.g. `us-gaap:` is optional, so you can use just the concept name like `Revenues`. Querying by concept does a partial regex match on the concept name `by_concept('RevenueFrom')` matches `us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax` and `us-gaap:RevenueFromContractWithCustomerTextBlock` Use `exact=True` to match the full concept name exactly. ### By Label You can filter facts by their labels, which are human-readable names associated with the concepts. `# Search by label text revenue_query = xb.query().by_label("Revenue")` To specify exact matches or partial matches, use the `exact` parameter: `sales_query = xb.query().by_label("Revenue", exact=False)` ### By Value `# Facts with values above $1 billion large_values = xb.query().by_value(lambda x: x > 1_000_000_000) # Facts within a range range_query = xb.query().by_value(lambda x: 100_000 <= x <= 1_000_000)` ### By Statement Type `# Facts from specific statements income_facts = xb.query().by_statement_type("IncomeStatement") balance_facts = xb.query().by_statement_type("BalanceSheet")` Method Chaining --------------- Combine multiple filters using method chaining: `# Complex query with multiple filters complex_query = (xbrl.query() .by_statement("IncomeStatement") .by_label("Revenue") .by_value(lambda x: x > 1_000_000) .sort_by('value', ascending=False) .limit(10)) results = complex_query.execute()` Data Transformations -------------------- ### Sorting `# Sort by value (descending) sorted_query = xbrl.query().sort_by('value', ascending=False) # Sort by concept name concept_sorted = xbrl.query().sort_by('concept')` ### Limiting Results `# Get top 10 results top_10 = xbrl.query().limit(10) # Pagination page_1 = xbrl.query().limit(20) page_2 = xbrl.query().offset(20).limit(20)` Working with Results -------------------- ### DataFrame Output `# Get specific columns df = query.to_dataframe('concept', 'label', 'value', 'period_end') # All available columns full_df = query.to_dataframe() # Column information print("Available columns:", df.columns.tolist())` ### Fact Structure Each fact contains the following key information: `fact = results[0] print(f"Concept: {fact['concept']}") print(f"Label: {fact['label']}") print(f"Value: {fact['value']}") print(f"Period: {fact['period_end']}") print(f"Units: {fact['units']}") print(f"Decimals: {fact['decimals']}")` Advanced Filtering ------------------ ### Dimensions `# Facts with specific dimensions dimensional_query = xbrl.query().by_dimension("ProductOrServiceAxis", "ProductMember") # Facts with any value for a dimension any_product_dim = xbrl.query().by_dimension("ProductOrServiceAxis") # Facts with NO dimensions (undimensioned facts) undimensioned_facts = xbrl.query().by_dimension(None) # Multiple dimensions multi_dim = xbrl.query().by_dimensions({ "ProductOrServiceAxis": "ProductMember", "GeographyAxis": "USMember" })` Performance Tips ---------------- 1. **Use specific filters**: Filter early to reduce data processing 2. **Limit results**: Use `.limit()` for large datasets 3. **Cache queries**: Store frequently used queries 4. **Select columns**: Use `to_dataframe()` with specific columns `# Efficient query pattern efficient_query = (xb.query() .by_statement("IncomeStatement") # Filter first .by_value(lambda x: x > 0) # Remove zeros .limit(100) # Limit results .to_dataframe('concept', 'value')) # Select columns` Examples -------- ### Finding Revenue Information `# All revenue-related facts revenue_facts = (xb.query() .by_label("revenue", exact=False) .sort_by('value', ascending=False) .execute()) for fact in revenue_facts: print(f"{fact['label']}: ${fact['value']:,}")` ### Comparing Quarterly Data `# Get quarterly revenue data quarterly_revenue = (xb.query() .by_concept("us-gaap:Revenues") .by_period_type("duration") .sort_by('period_end') .to_dataframe('period_end', 'value')) print(quarterly_revenue)` ### Balance Sheet Analysis `# Major balance sheet items balance_items = (xb.query() .by_statement("BalanceSheet") .by_value(lambda x: x > 1_000_000_000) # > $1B .sort_by('value', ascending=False) .to_dataframe('label', 'value')) print("Major Balance Sheet Items (> $1B):") print(balance_items)` This query system provides a flexible and powerful way to explore XBRL data, enabling detailed financial analysis and data extraction from individual filings. Back to top --- # Fund Shareholder Reports (N-CSR) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/fundshareholderreport-data-object-guide/#fund-shareholder-reports-n-csr-n-csrs-parse-certified-shareholder-reports-with-python) Fund Shareholder Reports (N-CSR / N-CSRS): Parse Certified Shareholder Reports with Python ========================================================================================== Registered investment companies file Form N-CSR annually and Form N-CSRS semiannually -- certified shareholder reports that include expense ratios, annual performance data, and top portfolio holdings for every share class. EdgarTools parses these Inline XBRL filings into structured Python objects using the OEF (Open-End Fund) taxonomy. `from edgar import get_filings filings = get_filings(form="N-CSR") report = filings[0].obj() report` ![Fund Shareholder Report N-CSR parsed with Python edgartools](https://edgartools.readthedocs.io/en/latest/images/fundshareholderreport-display.webp) Three lines to get a fully parsed certified shareholder report with fund name, net assets, expense ratios, and annual returns for every share class. * * * Access Expense Data ------------------- The `expense_data()` method returns a DataFrame with one row per share class, showing expense ratios and fees paid during the period: `report.expense_data()` | Column | What it is | | --- | --- | | `class_name` | Share class name (e.g., `"Investor Shares"`, `"Admiral Shares"`) | | `ticker` | Share class ticker symbol (if available) | | `expense_ratio_pct` | Total expense ratio as a decimal fraction | | `expenses_paid` | Total expenses paid in dollars during the period | | `advisory_fees_paid` | Advisory fees paid in dollars during the period | Expense ratios are stored as decimal fractions. An `expense_ratio_pct` of `0.0094` means a 0.94% expense ratio. Multiply by 100 to display as a percentage. * * * Access Performance Data ----------------------- The `performance_data()` method returns a DataFrame of average annual returns across all share classes and reporting periods: `report.performance_data()` | Column | What it is | | --- | --- | | `class_name` | Share class name | | `ticker` | Share class ticker symbol | | `period` | Return period label (e.g., `"1-Year"`, `"5-Year"`, `"10-Year"`) | | `return_pct` | Average annual return as a decimal fraction | | `inception_date` | Share class inception date (if reported) | Returns are decimal fractions. A `return_pct` of `0.1615` means a 16.15% return. Use this DataFrame to compare performance across share classes and time horizons in a single query. * * * Access Holdings Data -------------------- The `holdings_data()` method returns a DataFrame of top portfolio holdings disclosed in the report: `report.holdings_data()` | Column | What it is | | --- | --- | | `class_name` | Share class name | | `holding` | Holding name derived from the XBRL dimension member | | `pct_of_nav` | Holding as a percentage of net asset value (decimal fraction) | | `pct_of_total_inv` | Holding as a percentage of total investments (decimal fraction) | Holdings data availability varies significantly across filers. Some funds report detailed top-ten holdings; others omit this section entirely. Check `df.empty` before analysis. * * * Look Up a Specific Fund ----------------------- Search by management company name or CIK: `from edgar import Company company = Company("VANGUARD") filing = company.get_filings(form="N-CSR").latest(1) report = filing.obj() print(report.fund_name) # Fund name from the OEF taxonomy print(report.report_type) # "Annual" print(report.num_share_classes) # Number of share classes parsed print(report.net_assets) # Net assets (Decimal, or None)` A single N-CSR filing may cover multiple fund series within a complex. The `fund_name` property returns the first fund name found in the Inline XBRL document. * * * Access Semiannual Reports ------------------------- N-CSRS filings follow the same structure as N-CSR but cover the fund's semiannual period. The `is_annual` property distinguishes them: `from edgar import get_filings filings = get_filings(form="N-CSRS") report = filings[0].obj() print(report.report_type) # "Semi-Annual" print(report.is_annual) # False` Both form types return the same `FundShareholderReport` object. All three DataFrame methods work identically for both annual and semiannual reports. * * * Common Analysis Patterns ------------------------ ### Compare expense ratios across share classes `expenses = report.expense_data() # Display expense ratios as percentages expenses["expense_ratio_pct_display"] = expenses["expense_ratio_pct"] * 100 print(expenses[["class_name", "ticker", "expense_ratio_pct_display"]].to_string(index=False))` ### Sort share classes by long-term return `perf = report.performance_data() # Focus on 10-year returns ten_year = perf[perf["period"].str.contains("10", na=False)].copy() ten_year["return_display"] = ten_year["return_pct"] * 100 ten_year_sorted = ten_year.sort_values("return_display", ascending=False) print(ten_year_sorted[["class_name", "ticker", "return_display"]])` ### Check portfolio turnover `if report.portfolio_turnover is not None: turnover_pct = float(report.portfolio_turnover) * 100 print(f"Portfolio turnover: {turnover_pct:.1f}%") else: print("Portfolio turnover not reported")` ### Identify missing data before analysis `expenses = report.expense_data() performance = report.performance_data() holdings = report.holdings_data() for label, df in [("Expenses", expenses), ("Performance", performance), ("Holdings", holdings)]: if df.empty: print(f"{label}: not available in this filing") else: print(f"{label}: {len(df)} rows")` * * * Access Individual Objects ------------------------- The `share_classes` list provides direct access to each share class's `ShareClassInfo` object. These are useful for custom display, export, or filtering logic. ### Iterate over share classes `for sc in report.share_classes: ticker = f" ({sc.class_ticker})" if sc.class_ticker else "" print(f"{sc.class_name}{ticker}") print(f" Expense ratio: {float(sc.expense_ratio_pct) * 100:.2f}%" if sc.expense_ratio_pct else " Expense ratio: N/A") print(f" Holdings reported: {sc.holdings_count}" if sc.holdings_count else " Holdings count: N/A")` ### Inspect annual returns for a single class `sc = report.share_classes[0] for ret in sc.annual_returns: if ret.return_pct is not None: print(f" {ret.period_label}: {float(ret.return_pct) * 100:.2f}%")` ### Inspect top holdings for a single class `sc = report.share_classes[0] for holding in sc.holdings: if holding.pct_of_nav is not None: print(f" {holding.name}: {float(holding.pct_of_nav) * 100:.2f}% of NAV")` * * * Metadata Quick Reference ------------------------ | Property | Returns | Example | | --- | --- | --- | | `fund_name` | Fund name from OEF taxonomy | `"Vanguard 500 Index Fund"` | | `report_type` | `"Annual"` or `"Semi-Annual"` | `"Annual"` | | `is_annual` | Whether this is an N-CSR (annual) report | `True` | | `net_assets` | Net assets as `Decimal` (or `None`) | `Decimal("47382956000")` | | `portfolio_turnover` | Turnover rate as decimal fraction (or `None`) | `Decimal("0.7567")` | | `num_share_classes` | Number of share classes parsed | `3` | | `share_classes` | `List[ShareClassInfo]` for all share classes | Full per-class data | | `filing` | Source Filing object | `Filing` or `None` | | `cik` | CIK of the filer | `"0000102909"` | | `series_id` | SEC series ID | `"S000002277"` or `None` | * * * Methods Quick Reference ----------------------- | Method | Returns | What it does | | --- | --- | --- | | `expense_data()` | `DataFrame` | Expense ratios and fees for all share classes | | `performance_data()` | `DataFrame` | Average annual returns across all share classes and periods | | `holdings_data()` | `DataFrame` | Top holdings by percentage of NAV for all share classes | * * * Things to Know -------------- **Values are decimal fractions, not percentages.** Expense ratios, portfolio turnover, and return values are all stored as decimals. An `expense_ratio_pct` of `0.0094` means 0.94%. Multiply by 100 before displaying to users. **Net assets are in full dollars.** A `net_assets` value of `47382956000` means exactly $47.4 billion. There is no thousands scaling. **Annual and semiannual reports use the same object.** N-CSR (annual) and N-CSRS (semiannual) filings both produce a `FundShareholderReport`. Use `is_annual` to distinguish them. **XBRL is required.** `from_filing()` calls `filing.xbrl()` internally. If the filing does not contain Inline XBRL -- which is uncommon but possible for older filings -- `filing.obj()` returns `None`. Always check for `None` before accessing properties. **Ticker symbols are often absent.** The OEF taxonomy includes a `ClassTicker` concept, but many filers do not populate it. The `class_ticker` field will be `None` for most share classes from smaller fund complexes. **Holdings by NAV are sparsely populated.** The `pct_of_nav` field requires the filer to tag `oef:HoldingPctOfNav` against a `HoldingAxis` dimension. Many filers report only `holdings_count` (the total number of holdings) without disclosing individual holding percentages. **Multiple fund series per filing.** A single N-CSR filing can cover multiple series from a fund family. Only the first `oef:FundName` fact is captured in `fund_name`. The DataFrame methods aggregate data across all discovered share classes regardless of series. **Share class discovery uses ClassAxis dimensions.** Share classes are identified from the `oef:ClassAxis` dimension in the Inline XBRL. For single-class funds that do not use this dimension, a single placeholder class is constructed from undimensioned facts. **Approximately 6,600 filings per year.** N-CSR and N-CSRS together account for roughly 6,623 annual filings from registered investment companies, covering thousands of individual fund series. * * * Related ------- * [Fund Entities](https://edgartools.readthedocs.io/en/latest/guides/fund-entity-guide/) -- look up funds by ticker, navigate hierarchies * [Working with Filings](https://edgartools.readthedocs.io/en/latest/guides/working-with-filing/) -- general filing access patterns * [Fund Portfolios (N-PORT)](https://edgartools.readthedocs.io/en/latest/guides/nport-data-object-guide/) -- monthly fund portfolio holdings * [Money Market Funds (N-MFP)](https://edgartools.readthedocs.io/en/latest/guides/moneymarketfund-data-object-guide/) -- money market fund holdings and yields * [Fund Census (N-CEN)](https://edgartools.readthedocs.io/en/latest/guides/fundcensus-data-object-guide/) -- annual fund operational census with service provider data Back to top --- # Fund Portfolios (N-PORT) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/nport-data-object-guide/#n-port-parse-mutual-fund-portfolio-holdings-with-python) N-PORT: Parse Mutual Fund Portfolio Holdings with Python ======================================================== Every registered mutual fund and ETF files Form NPORT-P with the SEC each month, disclosing their complete portfolio -- every stock, bond, swap, and option they hold. EdgarTools parses these filings into structured Python objects so you can analyze fund portfolios in a few lines of code. `from edgar import get_filings filings = get_filings(form="NPORT-P") report = filings[0].obj() report` ![N-PORT mutual fund portfolio holdings parsed with Python edgartools](https://edgartools.readthedocs.io/en/stable/images/nport-p.webp) Three lines to get a fully parsed fund report with total assets, all investment positions, derivative exposures, and risk metrics. * * * Investment Data --------------- The `investment_data()` method returns a DataFrame with one row per position, sorted by absolute value: `report.investment_data()` | Column | What it is | | --- | --- | | `name` | Issuer name (`"APPLE INC"`) | | `title` | Security title (`"COMMON STOCK"`) | | `cusip` | 9-character CUSIP | | `ticker` | Resolved ticker symbol | | `balance` | Share count or par amount | | `units` | `"NS"` (shares), `"PA"` (principal), or `"NC"` (contracts) | | `value_usd` | Market value in **US dollars** | | `pct_value` | Percentage of net asset value | | `asset_category` | SEC category (`"EC"` equity, `"DBT"` debt, etc.) | | `issuer_category` | Issuer type (`"CORP"`, `"USG"`, `"MUN"`, etc.) | | `currency_code` | Currency of the position | | `investment_country` | Country code | | `is_derivative` | `True` for derivatives, `False` for securities | Values are in full US dollars -- unlike 13F filings, NPORT values are **not** in thousands. * * * Securities vs Derivatives ------------------------- Split the portfolio into traditional securities and derivative positions: `# Non-derivative positions only (stocks, bonds, etc.) report.securities_data() # Derivative positions only report.derivatives_data()` The `derivatives_data()` DataFrame adds columns specific to derivatives: | Column | What it is | | --- | --- | | `subtype` | Derivative subtype (e.g., `"CDS"`, `"IRS"`) | | `reference` | Reference entity or index | | `counterparty` | Counterparty name | | `notional_amount` | Notional value | | `unrealized_pnl` | Unrealized appreciation/depreciation | | `termination_date` | Contract expiration date | | `pct_value` | Percentage of NAV | See it live on edgar.tools The code above parses N-PORT portfolio holdings in Python. **edgar.tools** renders fund portfolios visually — browse holdings, derivative exposures, and risk metrics for any registered fund. * **[Browse company filings and fund data →](https://app.edgar.tools/companies?utm_source=edgartools-docs&utm_medium=see-live&utm_content=nport-guide) ** * **[Watch filings arrive in real time →](https://app.edgar.tools/filings?utm_source=edgartools-docs&utm_medium=see-live&utm_content=nport-guide) ** Free tier available. Also includes a REST API for programmatic access. [Pricing →](https://app.edgar.tools/pricing?utm_source=edgartools-docs&utm_medium=see-live&utm_content=nport-guide) * * * Derivative-Specific DataFrames ------------------------------ For detailed analysis of specific derivative types, use the dedicated methods. Each returns a DataFrame with type-specific fields: `report.swaps_data() # Interest rate swaps, CDS, total return swaps report.options_data() # Options with strike price, expiry, put/call report.forwards_data() # FX forwards with currencies and settlement report.futures_data() # Futures with payoff profile and expiry report.swaptions_data() # Swaption contracts` All derivative DataFrames share common fields (`name`, `title`, `counterparty`, `notional_amount`, `unrealized_pnl`) plus type-specific columns. For example, `swaps_data()` includes directional receive/pay legs, and `options_data()` includes `strike_price`, `written_or_purchased`, and `shares_per_contract`. * * * Fund Financials --------------- Access the fund's balance sheet data through `fund_info`: `print(f"Total assets: ${report.fund_info.total_assets:,.0f}") print(f"Total liabilities: ${report.fund_info.total_liabilities:,.0f}") print(f"Net assets: ${report.fund_info.net_assets:,.0f}")` * * * Risk Metrics ------------ ### Interest rate sensitivity NPORT filings include DV01 (dollar value of a 1 basis point move) and DV100 (dollar value of a 100 basis point move) across multiple time horizons: `for currency, metric in report.fund_info.current_metrics.items(): print(f"{currency} DV01 (1yr): {metric.intrstRtRiskdv01.period1Yr}") print(f"{currency} DV100 (1yr): {metric.intrstRtRiskdv100.period1Yr}")` ### Credit spread risk Investment grade and non-investment grade spread sensitivity, when reported: `ig = report.fund_info.credit_spread_risk_investment_grade if ig: print(f"IG spread risk (1yr): {ig.period1Yr}") nig = report.fund_info.credit_spread_risk_non_investment_grade if nig: print(f"Non-IG spread risk (1yr): {nig.period1Yr}")` * * * Look Up a Specific Fund ----------------------- Search for a fund by its CIK or by the management company: `from edgar import Company # By management company CIK vanguard = Company("0000102909") filing = vanguard.get_filings(form="NPORT-P").latest(1) report = filing.obj() print(report.general_info.name) # Management company print(report.general_info.series_name) # Specific fund/series print(report.reporting_period) # Report date` Or use the `Fund` class for a simpler path: `from edgar import Fund fund = Fund("VFINX") report = fund.get_latest_report() # Latest NPORT-P report df = fund.get_portfolio() # Portfolio as DataFrame` * * * Common Analysis Patterns ------------------------ ### Top holdings by value `df = report.investment_data(include_derivatives=False) total = df['value_usd'].sum() df['weight'] = (df['value_usd'] / total * 100).round(2) df[['name', 'ticker', 'value_usd', 'weight']].head(10)` ### Sector allocation `df = report.securities_data() df.groupby('asset_category')['value_usd'].sum().sort_values(ascending=False)` ### Derivative exposure as percentage of NAV `net_assets = report.fund_info.net_assets deriv = report.derivatives_data() if not deriv.empty: total_notional = deriv['notional_amount'].abs().sum() print(f"Derivative notional: ${total_notional:,.0f}") print(f"As % of NAV: {total_notional / net_assets * 100:.1f}%")` * * * Metadata Quick Reference ------------------------ | Property | Returns | Example | | --- | --- | --- | | `general_info.name` | Management company | `"VANGUARD CHESTER FUNDS"` | | `general_info.series_name` | Fund/series name | `"Vanguard Target Retirement 2035 Fund"` | | `general_info.cik` | SEC CIK | `"0000102909"` | | `general_info.rep_period_date` | Report date | `"2024-03-31"` | | `general_info.fiscal_year_end` | Fiscal year end | `"0131"` | | `general_info.series_id` | SEC series ID | `"S000004104"` | | `general_info.reg_lei` | LEI | `"549300..."` | | `reporting_period` | Same as rep\_period\_date | `"2024-03-31"` | | `name` | Company - Series | `"VANGUARD CHESTER FUNDS - Vanguard Target..."` | | `has_investments` | Has any positions? | `True` | | `header.submission_type` | Form type filed | `"NPORT-P"` | | `filing` | Source Filing object | `Filing` or `None` | | `cik` | CIK of the filer | `"0000102909"` | | `series_id` | SEC series ID | `"S000004104"` or `None` | * * * Methods Quick Reference ----------------------- | Call | Returns | What it does | | --- | --- | --- | | `report.investment_data()` | `DataFrame` | All positions (securities + derivatives) | | `report.securities_data()` | `DataFrame` | Non-derivative positions only | | `report.derivatives_data()` | `DataFrame` | Derivative positions with P&L | | `report.swaps_data()` | `DataFrame` | Swap details with receive/pay legs | | `report.options_data()` | `DataFrame` | Options with strike, expiry, put/call | | `report.forwards_data()` | `DataFrame` | FX forwards with settlement info | | `report.futures_data()` | `DataFrame` | Futures contracts | | `report.swaptions_data()` | `DataFrame` | Swaption contracts | | `report.derivatives` | `list` | Raw derivative investment objects | | `report.non_derivatives` | `list` | Raw non-derivative investment objects | | `report.get_fund_series()` | `FundSeries` | Fund series object | | `report.get_ticker_for_series()` | `str` | Ticker for this series | | `report.get_tickers_for_series()` | `list[str]` | All tickers for this series | | `report.matches_ticker(ticker)` | `bool` | Whether report matches a ticker | * * * Things to Know -------------- **Values are in full dollars.** Unlike 13F filings (which report in thousands), NPORT values are in actual USD. A `value_usd` of 135,364,000 means exactly $135.4 million. **NPORT-P vs NPORT-EX.** NPORT-P is the full monthly filing with all positions and risk metrics. NPORT-EX is a quarterly exhibit containing only a subset of holdings. EdgarTools handles both, but NPORT-P has the most complete data. **Monthly reporting.** Funds file NPORT-P every month, but only the quarter-end filings are made public. The other two months in each quarter are confidential (released after 60 days). **Derivative coverage.** EdgarTools parses all five derivative types: swaps, options, forwards, futures, and swaptions. Each type has a dedicated `*_data()` method for detailed analysis. **Ticker resolution.** Tickers are resolved from CUSIPs and fund metadata. Most common securities resolve correctly, but obscure or private placements may have blank tickers. **Empty DataFrames.** Methods like `derivatives_data()` and `swaps_data()` return an empty DataFrame when no positions of that type exist. Always check with `df.empty` before analysis. * * * Related ------- * [Fund Entities](https://edgartools.readthedocs.io/en/stable/guides/fund-entity-guide/) -- look up funds by ticker, navigate hierarchies * [Working with Filings](https://edgartools.readthedocs.io/en/stable/guides/working-with-filing/) -- general filing access patterns Back to top --- # Money Market Funds (N-MFP) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/moneymarketfund-data-object-guide/#n-mfp-parse-money-market-fund-filings-with-python) N-MFP: Parse Money Market Fund Filings with Python ================================================== Money market funds file Form N-MFP monthly to report their complete portfolios, yields, net asset values, and liquidity metrics. EdgarTools parses both N-MFP3 (June 2024+) and N-MFP2 (2010–mid 2024) filings into structured Python objects. `from edgar import get_filings filings = get_filings(form="N-MFP3") mmf = filings[0].obj() mmf` ![Money market fund parsed with Python edgartools](https://edgartools.readthedocs.io/en/latest/images/moneymarketfund-display.webp) Three lines to get a fully parsed money market fund report with net assets, weighted average maturity, portfolio holdings, and share class details. * * * Portfolio Holdings ------------------ The `portfolio_data()` method returns a DataFrame with every security in the fund, sorted by market value: `mmf.portfolio_data()` ![Money market fund portfolio holdings table](https://edgartools.readthedocs.io/en/latest/images/moneymarketfund-portfolio.webp) | Column | What it is | | --- | --- | | `issuer` | Issuer name (`"U.S. Treasury"`) | | `title` | Security title (`"T-Bill"`) | | `cusip` | 9-character CUSIP | | `isin` | ISIN identifier | | `category` | Investment category (`"TreasuryDebt"`, `"CommPaper"`, etc.) | | `maturity_wam` | Maturity date for WAM calculation | | `maturity_wal` | Maturity date for WAL calculation | | `yield` | Yield rate as decimal | | `market_value` | Market value in **full dollars** | | `amortized_cost` | Amortized cost in full dollars | | `pct_of_nav` | Percentage of net asset value | | `daily_liquid` | Daily liquid asset flag | | `weekly_liquid` | Weekly liquid asset flag | | `has_repo` | Has repurchase agreement collateral | Market values are in full dollars -- not thousands. A `market_value` of 5,000,000,000 means exactly $5 billion. * * * Repurchase Agreement Collateral ------------------------------- Money market funds often hold repurchase agreements secured by government or agency securities. The `collateral_data()` method flattens all repo collateral into a single DataFrame: `mmf.collateral_data()` | Column | What it is | | --- | --- | | `security_issuer` | The repo counterparty | | `security_cusip` | CUSIP of the repo agreement | | `collateral_issuer` | Issuer of the collateral security | | `collateral_cusip` | CUSIP of the collateral | | `collateral_lei` | LEI of collateral issuer | | `maturity_date` | Collateral maturity date | | `coupon` | Collateral coupon rate | | `principal_amount` | Principal amount of collateral | | `collateral_value` | Market value of collateral | | `collateral_category` | Type of collateral security | This is useful for analyzing the credit quality and composition of repo collateral backing the fund's assets. * * * Share Class Information ----------------------- Money market funds typically offer multiple share classes with different expense structures and minimum investments. Access share class details with: `mmf.share_class_data()` | Column | What it is | | --- | --- | | `class_name` | Share class name (`"Class A"`, `"Institutional"`) | | `class_id` | SEC series identifier | | `min_investment` | Minimum initial investment | | `net_assets` | Net assets for this share class | | `shares_outstanding` | Total shares outstanding | * * * Yield and NAV Time Series ------------------------- ### N-MFP3 (June 2024+): Daily Time Series N-MFP3 filings include 20 days of daily data for yields, NAV, and liquidity metrics. `# 7-day gross yield history (series-level) mmf.yield_history() # Daily NAV per share history (series-level) mmf.nav_history() # Daily and weekly liquid asset percentages mmf.liquidity_history()` Each method returns a DataFrame with a `date` column and the corresponding metric values. **Yield history** shows the 7-day gross yield over the reporting period. **NAV history** tracks daily NAV per share. **Liquidity history** shows both daily liquid assets (securities that can convert to cash in 1 business day) and weekly liquid assets (convertible within 5 business days), reported as both dollar amounts and percentages of net assets. ### N-MFP2 (2010–mid 2024): Weekly Snapshots N-MFP2 filings report weekly Friday snapshots instead of daily time series. The same methods work, but the `date` column uses labels like `"week_1"`, `"week_2"`, etc., and yields are single scalar values rather than a time series. * * * Holdings by Investment Category ------------------------------- Group holdings by SEC investment category to see asset class allocation: `mmf.holdings_by_category()` ![Money market fund holdings grouped by category](https://edgartools.readthedocs.io/en/latest/images/moneymarketfund-categories.webp) Returns a DataFrame with columns: | Column | What it is | | --- | --- | | `category` | Investment category (e.g., `"TreasuryDebt"`, `"CommPaper"`) | | `count` | Number of securities in this category | | `total_market_value` | Sum of market values | | `total_pct` | Percentage of fund NAV | Sorted by total market value descending, making it easy to identify the fund's largest asset class exposures. * * * Look Up a Specific Fund ----------------------- Search for a fund by the management company CIK or name: `from edgar import Company # By management company name or CIK vanguard = Company("VANGUARD") filing = vanguard.get_filings(form="N-MFP3").latest(1) mmf = filing.obj() print(mmf.name) # Fund series name print(mmf.report_date) # Reporting period end date print(f"${mmf.net_assets:,.0f}") # Net assets` Or use the `Fund` class for a simpler path: `from edgar import Fund fund = Fund("VMFXX") mmf = fund.get_latest_report(form="N-MFP3")` The `name` property returns the full series name (e.g., "Vanguard Prime Money Market Fund"), and `report_date` shows the period end date. * * * Common Analysis Patterns ------------------------ ### Weighted average maturity and credit quality `print(f"WAM: {mmf.average_maturity_wam} days") print(f"WAL: {mmf.average_maturity_wal} days") print(f"Fund category: {mmf.fund_category}")` Weighted Average Maturity (WAM) and Weighted Average Life (WAL) are key metrics for assessing interest rate sensitivity. Fund category indicates the type of money market fund (e.g., "Government", "Prime", "Tax-Exempt"). ### Top holdings concentration `holdings = mmf.portfolio_data() # The pct_of_nav column already contains percentages holdings[['issuer', 'title', 'market_value', 'pct_of_nav']].head(10) # Or calculate manually if needed import pandas as pd holdings['market_value_float'] = holdings['market_value'].astype(float) total = holdings['market_value_float'].sum() holdings['weight_pct'] = (holdings['market_value_float'] / total * 100).round(2)` ### Treasury vs. agency vs. repo exposure `by_category = mmf.holdings_by_category() treasury = by_category[by_category['category'] == 'TreasuryDebt']['total_pct'].sum() agency = by_category[by_category['category'] == 'AgencyDebt']['total_pct'].sum() repo = by_category[by_category['category'] == 'Repo']['total_pct'].sum() print(f"Treasury: {treasury:.1f}%") print(f"Agency: {agency:.1f}%") print(f"Repo: {repo:.1f}%")` ### Liquidity buffer analysis `liq = mmf.liquidity_history() if not liq.empty: latest = liq.iloc[-1] print(f"Daily liquid assets: {latest['pct_daily_liquid']:.1f}%") print(f"Weekly liquid assets: {latest['pct_weekly_liquid']:.1f}%")` SEC rules require money market funds to maintain minimum liquidity levels. Daily liquid assets must be at least 10% of total assets, and weekly liquid assets must be at least 30%. * * * Metadata Quick Reference ------------------------ | Property | Returns | Example | | --- | --- | --- | | `name` | Fund series name | `"Vanguard Prime Money Market Fund"` | | `report_date` | Reporting period end | `"2024-10-31"` | | `fund_category` | Fund type | `"Prime"` | | `net_assets` | Net assets (Decimal) | `Decimal('26435168844.97')` | | `num_securities` | Number of holdings | `92` | | `num_share_classes` | Number of share classes | `4` | | `average_maturity_wam` | WAM in days | `45` | | `average_maturity_wal` | WAL in days | `52` | | `filing` | Source Filing object | `Filing` or `None` | | `cik` | CIK of the filer | `"0000102909"` | | `series_id` | SEC series ID | `"S000004104"` or `None` | * * * Methods Quick Reference ----------------------- | Call | Returns | What it does | | --- | --- | --- | | `mmf.portfolio_data()` | `DataFrame` | All securities sorted by market value | | `mmf.share_class_data()` | `DataFrame` | Share class details and net assets | | `mmf.yield_history()` | `DataFrame` | 7-day gross yield time series | | `mmf.nav_history()` | `DataFrame` | Daily NAV per share time series | | `mmf.liquidity_history()` | `DataFrame` | Daily/weekly liquid asset metrics | | `mmf.collateral_data()` | `DataFrame` | Repo collateral details flattened | | `mmf.holdings_by_category()` | `DataFrame` | Holdings grouped by investment category | * * * Things to Know -------------- **Values are in full dollars.** Unlike 13F filings (which report in thousands), N-MFP market values are in actual USD. A `market_value` of 5,000,000,000 means exactly $5 billion. **N-MFP3 vs N-MFP2.** N-MFP3 (June 2024+) includes daily time series with dates for the past 20 business days. N-MFP2 (2010–mid 2024) includes weekly Friday snapshots without explicit dates. EdgarTools parses both transparently. **Time series data.** For N-MFP3, `yield_history()`, `nav_history()`, and `liquidity_history()` return DataFrames with 20 daily observations. For N-MFP2, they return weekly snapshots labeled `"week_1"` through `"week_5"`. **Repo collateral.** Repurchase agreements are a common money market fund investment. Use `collateral_data()` to analyze what securities back these repos. The `has_repo` column in `portfolio_data()` indicates which securities have collateral details. **Monthly filings, quarterly public disclosure.** Funds file monthly, but only quarter-end filings are immediately public. Mid-quarter filings are released with a 60-day delay. **Share class structure.** The same underlying portfolio may be divided into multiple share classes with different expense ratios and minimum investments. `share_class_data()` shows this structure. **WAM and WAL.** Weighted Average Maturity (WAM) uses the next interest rate reset date for floating-rate securities. Weighted Average Life (WAL) uses final maturity. WAL is always ≥ WAM. * * * Related ------- * [Fund Entities](https://edgartools.readthedocs.io/en/latest/guides/fund-entity-guide/) -- look up funds by ticker, navigate hierarchies * [Working with Filings](https://edgartools.readthedocs.io/en/latest/guides/working-with-filing/) -- general filing access patterns * [Fund Portfolios (N-PORT)](https://edgartools.readthedocs.io/en/latest/guides/nport-data-object-guide/) -- mutual fund and ETF portfolio holdings Back to top --- # Money Market Funds (N-MFP) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/moneymarketfund-data-object-guide/#n-mfp-parse-money-market-fund-filings-with-python) N-MFP: Parse Money Market Fund Filings with Python ================================================== Money market funds file Form N-MFP monthly to report their complete portfolios, yields, net asset values, and liquidity metrics. EdgarTools parses both N-MFP3 (June 2024+) and N-MFP2 (2010–mid 2024) filings into structured Python objects. `from edgar import get_filings filings = get_filings(form="N-MFP3") mmf = filings[0].obj() mmf` ![Money market fund parsed with Python edgartools](https://edgartools.readthedocs.io/en/stable/images/moneymarketfund-display.webp) Three lines to get a fully parsed money market fund report with net assets, weighted average maturity, portfolio holdings, and share class details. * * * Portfolio Holdings ------------------ The `portfolio_data()` method returns a DataFrame with every security in the fund, sorted by market value: `mmf.portfolio_data()` ![Money market fund portfolio holdings table](https://edgartools.readthedocs.io/en/stable/images/moneymarketfund-portfolio.webp) | Column | What it is | | --- | --- | | `issuer` | Issuer name (`"U.S. Treasury"`) | | `title` | Security title (`"T-Bill"`) | | `cusip` | 9-character CUSIP | | `isin` | ISIN identifier | | `category` | Investment category (`"TreasuryDebt"`, `"CommPaper"`, etc.) | | `maturity_wam` | Maturity date for WAM calculation | | `maturity_wal` | Maturity date for WAL calculation | | `yield` | Yield rate as decimal | | `market_value` | Market value in **full dollars** | | `amortized_cost` | Amortized cost in full dollars | | `pct_of_nav` | Percentage of net asset value | | `daily_liquid` | Daily liquid asset flag | | `weekly_liquid` | Weekly liquid asset flag | | `has_repo` | Has repurchase agreement collateral | Market values are in full dollars -- not thousands. A `market_value` of 5,000,000,000 means exactly $5 billion. * * * Repurchase Agreement Collateral ------------------------------- Money market funds often hold repurchase agreements secured by government or agency securities. The `collateral_data()` method flattens all repo collateral into a single DataFrame: `mmf.collateral_data()` | Column | What it is | | --- | --- | | `security_issuer` | The repo counterparty | | `security_cusip` | CUSIP of the repo agreement | | `collateral_issuer` | Issuer of the collateral security | | `collateral_cusip` | CUSIP of the collateral | | `collateral_lei` | LEI of collateral issuer | | `maturity_date` | Collateral maturity date | | `coupon` | Collateral coupon rate | | `principal_amount` | Principal amount of collateral | | `collateral_value` | Market value of collateral | | `collateral_category` | Type of collateral security | This is useful for analyzing the credit quality and composition of repo collateral backing the fund's assets. * * * Share Class Information ----------------------- Money market funds typically offer multiple share classes with different expense structures and minimum investments. Access share class details with: `mmf.share_class_data()` | Column | What it is | | --- | --- | | `class_name` | Share class name (`"Class A"`, `"Institutional"`) | | `class_id` | SEC series identifier | | `min_investment` | Minimum initial investment | | `net_assets` | Net assets for this share class | | `shares_outstanding` | Total shares outstanding | * * * Yield and NAV Time Series ------------------------- ### N-MFP3 (June 2024+): Daily Time Series N-MFP3 filings include 20 days of daily data for yields, NAV, and liquidity metrics. `# 7-day gross yield history (series-level) mmf.yield_history() # Daily NAV per share history (series-level) mmf.nav_history() # Daily and weekly liquid asset percentages mmf.liquidity_history()` Each method returns a DataFrame with a `date` column and the corresponding metric values. **Yield history** shows the 7-day gross yield over the reporting period. **NAV history** tracks daily NAV per share. **Liquidity history** shows both daily liquid assets (securities that can convert to cash in 1 business day) and weekly liquid assets (convertible within 5 business days), reported as both dollar amounts and percentages of net assets. ### N-MFP2 (2010–mid 2024): Weekly Snapshots N-MFP2 filings report weekly Friday snapshots instead of daily time series. The same methods work, but the `date` column uses labels like `"week_1"`, `"week_2"`, etc., and yields are single scalar values rather than a time series. * * * Holdings by Investment Category ------------------------------- Group holdings by SEC investment category to see asset class allocation: `mmf.holdings_by_category()` ![Money market fund holdings grouped by category](https://edgartools.readthedocs.io/en/stable/images/moneymarketfund-categories.webp) Returns a DataFrame with columns: | Column | What it is | | --- | --- | | `category` | Investment category (e.g., `"TreasuryDebt"`, `"CommPaper"`) | | `count` | Number of securities in this category | | `total_market_value` | Sum of market values | | `total_pct` | Percentage of fund NAV | Sorted by total market value descending, making it easy to identify the fund's largest asset class exposures. * * * Look Up a Specific Fund ----------------------- Search for a fund by the management company CIK or name: `from edgar import Company # By management company name or CIK vanguard = Company("VANGUARD") filing = vanguard.get_filings(form="N-MFP3").latest(1) mmf = filing.obj() print(mmf.name) # Fund series name print(mmf.report_date) # Reporting period end date print(f"${mmf.net_assets:,.0f}") # Net assets` Or use the `Fund` class for a simpler path: `from edgar import Fund fund = Fund("VMFXX") mmf = fund.get_latest_report(form="N-MFP3")` The `name` property returns the full series name (e.g., "Vanguard Prime Money Market Fund"), and `report_date` shows the period end date. * * * Common Analysis Patterns ------------------------ ### Weighted average maturity and credit quality `print(f"WAM: {mmf.average_maturity_wam} days") print(f"WAL: {mmf.average_maturity_wal} days") print(f"Fund category: {mmf.fund_category}")` Weighted Average Maturity (WAM) and Weighted Average Life (WAL) are key metrics for assessing interest rate sensitivity. Fund category indicates the type of money market fund (e.g., "Government", "Prime", "Tax-Exempt"). ### Top holdings concentration `holdings = mmf.portfolio_data() # The pct_of_nav column already contains percentages holdings[['issuer', 'title', 'market_value', 'pct_of_nav']].head(10) # Or calculate manually if needed import pandas as pd holdings['market_value_float'] = holdings['market_value'].astype(float) total = holdings['market_value_float'].sum() holdings['weight_pct'] = (holdings['market_value_float'] / total * 100).round(2)` ### Treasury vs. agency vs. repo exposure `by_category = mmf.holdings_by_category() treasury = by_category[by_category['category'] == 'TreasuryDebt']['total_pct'].sum() agency = by_category[by_category['category'] == 'AgencyDebt']['total_pct'].sum() repo = by_category[by_category['category'] == 'Repo']['total_pct'].sum() print(f"Treasury: {treasury:.1f}%") print(f"Agency: {agency:.1f}%") print(f"Repo: {repo:.1f}%")` ### Liquidity buffer analysis `liq = mmf.liquidity_history() if not liq.empty: latest = liq.iloc[-1] print(f"Daily liquid assets: {latest['pct_daily_liquid']:.1f}%") print(f"Weekly liquid assets: {latest['pct_weekly_liquid']:.1f}%")` SEC rules require money market funds to maintain minimum liquidity levels. Daily liquid assets must be at least 10% of total assets, and weekly liquid assets must be at least 30%. * * * Metadata Quick Reference ------------------------ | Property | Returns | Example | | --- | --- | --- | | `name` | Fund series name | `"Vanguard Prime Money Market Fund"` | | `report_date` | Reporting period end | `"2024-10-31"` | | `fund_category` | Fund type | `"Prime"` | | `net_assets` | Net assets (Decimal) | `Decimal('26435168844.97')` | | `num_securities` | Number of holdings | `92` | | `num_share_classes` | Number of share classes | `4` | | `average_maturity_wam` | WAM in days | `45` | | `average_maturity_wal` | WAL in days | `52` | | `filing` | Source Filing object | `Filing` or `None` | | `cik` | CIK of the filer | `"0000102909"` | | `series_id` | SEC series ID | `"S000004104"` or `None` | * * * Methods Quick Reference ----------------------- | Call | Returns | What it does | | --- | --- | --- | | `mmf.portfolio_data()` | `DataFrame` | All securities sorted by market value | | `mmf.share_class_data()` | `DataFrame` | Share class details and net assets | | `mmf.yield_history()` | `DataFrame` | 7-day gross yield time series | | `mmf.nav_history()` | `DataFrame` | Daily NAV per share time series | | `mmf.liquidity_history()` | `DataFrame` | Daily/weekly liquid asset metrics | | `mmf.collateral_data()` | `DataFrame` | Repo collateral details flattened | | `mmf.holdings_by_category()` | `DataFrame` | Holdings grouped by investment category | * * * Things to Know -------------- **Values are in full dollars.** Unlike 13F filings (which report in thousands), N-MFP market values are in actual USD. A `market_value` of 5,000,000,000 means exactly $5 billion. **N-MFP3 vs N-MFP2.** N-MFP3 (June 2024+) includes daily time series with dates for the past 20 business days. N-MFP2 (2010–mid 2024) includes weekly Friday snapshots without explicit dates. EdgarTools parses both transparently. **Time series data.** For N-MFP3, `yield_history()`, `nav_history()`, and `liquidity_history()` return DataFrames with 20 daily observations. For N-MFP2, they return weekly snapshots labeled `"week_1"` through `"week_5"`. **Repo collateral.** Repurchase agreements are a common money market fund investment. Use `collateral_data()` to analyze what securities back these repos. The `has_repo` column in `portfolio_data()` indicates which securities have collateral details. **Monthly filings, quarterly public disclosure.** Funds file monthly, but only quarter-end filings are immediately public. Mid-quarter filings are released with a 60-day delay. **Share class structure.** The same underlying portfolio may be divided into multiple share classes with different expense ratios and minimum investments. `share_class_data()` shows this structure. **WAM and WAL.** Weighted Average Maturity (WAM) uses the next interest rate reset date for floating-rate securities. Weighted Average Life (WAL) uses final maturity. WAL is always ≥ WAM. * * * Related ------- * [Fund Entities](https://edgartools.readthedocs.io/en/stable/guides/fund-entity-guide/) -- look up funds by ticker, navigate hierarchies * [Working with Filings](https://edgartools.readthedocs.io/en/stable/guides/working-with-filing/) -- general filing access patterns * [Fund Portfolios (N-PORT)](https://edgartools.readthedocs.io/en/stable/guides/nport-data-object-guide/) -- mutual fund and ETF portfolio holdings Back to top --- # Fund Census (N-CEN) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/fundcensus-data-object-guide/#fund-census-n-cen-parse-annual-fund-reports-with-python) Fund Census (N-CEN): Parse Annual Fund Reports with Python ========================================================== Every registered investment company files Form N-CEN annually -- a comprehensive census of fund operational data. EdgarTools parses these filings into structured Python objects, revealing fund series, service providers, board composition, ETF mechanics, broker relationships, and securities lending programs. `from edgar import get_filings filings = get_filings(form="N-CEN") census = filings[0].obj() census` ![Fund Census N-CEN report parsed with Python edgartools](https://edgartools.readthedocs.io/en/latest/images/fundcensus-display.webp) Three lines to get a fully parsed annual census with registrant info, fund series, directors, and all operational disclosures. * * * Access Series Data ------------------ The `series_data()` method returns a DataFrame with one row per fund series, showing assets, commission totals, and service provider counts: `census.series_data()` | Column | What it is | | --- | --- | | `name` | Fund series name | | `series_id` | SEC series identifier (e.g., `"S000052123"`) | | `lei` | Legal Entity Identifier | | `fund_type` | Fund classification | | `avg_net_assets` | Average monthly net assets | | `aggregate_commission` | Total broker commissions paid | | `num_advisers` | Count of investment advisers | | `num_custodians` | Count of custodians | | `has_etf` | Whether series has ETF structure | Average net assets and commission figures are in full dollars (not thousands). An `avg_net_assets` of 19,574,292 means exactly $19.6 million. * * * Access Service Provider Network ------------------------------- See who manages, custodies, administers, and services the funds: `census.service_providers()` Returns a DataFrame with all service providers flattened across fund series: | Column | What it is | | --- | --- | | `series_name` | Fund series name | | `series_id` | Series identifier | | `role` | Provider role (`"adviser"`, `"custodian"`, `"transfer agent"`, etc.) | | `provider_name` | Provider firm name | | `lei` | Legal Entity Identifier | | `affiliated` | Whether provider is affiliated with the fund | This includes advisers, custodians, transfer agents, administrators, pricing services, and shareholder servicing agents -- everyone touching the fund's operations. * * * Analyze Broker Relationships ---------------------------- N-CEN discloses broker-dealer and broker commission payments: `census.broker_data()` | Column | What it is | | --- | --- | | `series_name` | Fund series name | | `series_id` | Series identifier | | `type` | `"broker-dealer"` or `"broker"` | | `broker_name` | Firm name | | `lei` | Legal Entity Identifier | | `commission` | Commission amount paid | Commission values are in full dollars. Use this to analyze broker concentration and trading costs across the fund complex. * * * Examine Board Composition ------------------------- Access the board of directors with interested-person flags: `census.director_data()` | Column | What it is | | --- | --- | | `name` | Director name | | `crd_number` | CRD identifier (if registered) | | `interested_person` | Whether director is an interested person under the '40 Act | Directors marked as "interested persons" have material relationships with the fund or its adviser. Independent directors are not interested persons. * * * Access ETF-Specific Data ------------------------ For funds with ETF structures, get creation/redemption mechanics and authorized participants: `census.etf_data()` Returns a DataFrame with ETF-specific metrics: | Column | What it is | | --- | --- | | `series_name` | Fund series name | | `series_id` | Series identifier | | `exchange` | Primary exchange (e.g., `"NYSE"`, `"NASDAQ"`) | | `ticker` | Trading symbol | | `creation_unit_size` | Number of shares per creation unit | | `avg_pct_purchased_in_kind` | Average percentage of creations settled in-kind | | `avg_pct_redeemed_in_kind` | Average percentage of redemptions settled in-kind | | `is_in_kind` | Whether fund permits in-kind transactions | | `num_authorized_participants` | Count of authorized participants | This DataFrame is empty for fund complexes without ETFs. Always check with `df.empty` before analysis. * * * Look Up a Specific Fund Complex ------------------------------- Search by management company CIK or name: `from edgar import Company company = Company("VANGUARD") filing = company.get_filings(form="N-CEN").latest(1) census = filing.obj() print(census.name) # Registrant name print(f"{census.num_series} series") # Series count in this filing print(census.report_date) # Report period end` A single N-CEN filing may cover multiple fund series, or just one series from a larger complex. Check `census.total_series` to see how many series the registrant has total. * * * Common Analysis Patterns ------------------------ ### Service provider concentration `providers = census.service_providers() # Count unique providers per role providers.groupby('role')['provider_name'].nunique() # Find affiliated providers affiliated = providers[providers['affiliated'] == True]` ### Commission analysis `brokers = census.broker_data() if not brokers.empty: # Top brokers by commission top_brokers = brokers.groupby('broker_name')['commission'].sum().sort_values(ascending=False) print(top_brokers.head(10))` ### Board independence `directors = census.director_data() total = len(directors) interested = directors['interested_person'].sum() independent = total - interested print(f"Board composition: {independent} independent, {interested} interested")` ### ETF creation mechanics `etf = census.etf_data() if not etf.empty: for idx, row in etf.iterrows(): print(f"{row['ticker']}: {row['avg_pct_purchased_in_kind']:.1f}% in-kind purchases")` * * * Access Individual Objects ------------------------- The FundCensus class uses Pydantic models for structured access to all fund census data. These are useful when building custom applications or exporting to other formats. ### Registrant Information `reg = census.registrant print(reg.name) # Registrant name print(reg.cik) # CIK print(reg.classification_type) # Fund classification print(reg.total_series) # Total series count print(reg.accountant.name) # Public accountant print(reg.cco_name) # Chief Compliance Officer` ### Fund Series `for series in census.series: print(f"{series.name} ({series.series_id})") print(f" Advisers: {len(series.advisers)}") print(f" Custodians: {len(series.custodians)}") print(f" Securities lending: {series.is_securities_lending}") # ETF details if series.etf_info: etf = series.etf_info print(f" ETF ticker: {etf.ticker}") print(f" Exchange: {etf.exchange}") print(f" Creation unit: {etf.creation_unit_size}")` ### Service Providers `series = census.series[0] # Investment advisers for adviser in series.advisers: print(f"{adviser.name} - {adviser.role}") print(f" Affiliated: {adviser.is_affiliated}") # Custodians for custodian in series.custodians: print(f"{custodian.name}")` ### Broker Dealers `series = census.series[0] for bd in series.broker_dealers: print(f"{bd.name}") print(f" Commission: ${bd.commission:,.2f}" if bd.commission else " No commission reported") print(f" LEI: {bd.lei}")` ### Line of Credit `series = census.series[0] if series.line_of_credit: loc = series.line_of_credit if loc.has_line_of_credit: print(f"Line of credit: ${loc.size:,.0f}") print(f"Committed: {loc.is_committed}") print(f"Institutions: {', '.join(loc.institution_names)}")` ### Securities Lending `series = census.series[0] for sl in series.securities_lending: print(f"Agent: {sl.agent_name}") print(f" Affiliated: {sl.is_affiliated}") print(f" Indemnified: {sl.is_indemnified}")` * * * Metadata Quick Reference ------------------------ | Property | Returns | Example | | --- | --- | --- | | `name` | Registrant name | `"360 Funds"` | | `cik` | CIK identifier | `"0001319067"` | | `report_date` | Report period end date | `"2025-11-30"` | | `num_series` | Number of series in filing | `1` | | `total_series` | Total series registered | `8` | | `classification_type` | Fund classification | `"N-1A"` | | `is_etf_company` | Has any ETF series | `True` or `False` | | `registrant` | RegistrantInfo object | Full registrant details | | `series` | List of FundSeriesInfo | All series with full details | | `filing` | Source Filing object | `Filing` or `None` | * * * Methods Quick Reference ----------------------- | Method | Returns | What it does | | --- | --- | --- | | `series_data()` | `DataFrame` | Fund series summary with assets and counts | | `service_providers()` | `DataFrame` | All service providers flattened across series | | `broker_data()` | `DataFrame` | Broker-dealer and broker commission data | | `director_data()` | `DataFrame` | Board of directors with interested-person flags | | `etf_data()` | `DataFrame` | ETF-specific metrics for ETF series | * * * Things to Know -------------- **Values are in full dollars.** Unlike 13F filings (which report in thousands), N-CEN values are in actual USD. An `avg_net_assets` of 19,574,292 means exactly $19.6 million. **Annual filings.** N-CEN is filed once per year, unlike monthly N-PORT or N-MFP reports. Each filing covers the fund's fiscal year. **Multiple series.** A single N-CEN filing may cover one series or dozens, depending on how the fund complex structures its filings. Check `num_series` vs `total_series` to understand coverage. **N/A sentinel values.** The XML uses "N/A" as a sentinel. EdgarTools converts these to `None` automatically. **Boolean display.** Pydantic booleans render as "Yes/No" in the Rich display for readability. **Service provider diversity.** Funds use many service provider types: advisers, custodians, transfer agents, administrators, pricing services, shareholder servicing agents. All are captured in `service_providers()`. **ETF mechanics.** Only series with ETF structures have `etf_info` populated. Use `etf_data()` to filter to ETF series automatically. **Securities lending.** Not all funds engage in securities lending. Check `is_securities_lending` or the length of `series.securities_lending`. **Line of credit.** Funds may have committed or uncommitted lines of credit with one or more institutions. The `line_of_credit` object captures all details. **Director CRD numbers.** Only registered directors have CRD numbers. Many directors won't have one -- this is normal. * * * Related ------- * [Fund Entities](https://edgartools.readthedocs.io/en/latest/guides/fund-entity-guide/) -- look up funds by ticker, navigate hierarchies * [Working with Filings](https://edgartools.readthedocs.io/en/latest/guides/working-with-filing/) -- general filing access patterns * [Fund Portfolios (N-PORT)](https://edgartools.readthedocs.io/en/latest/guides/nport-data-object-guide/) -- monthly fund portfolio holdings * [Money Market Funds (N-MFP)](https://edgartools.readthedocs.io/en/latest/guides/moneymarketfund-data-object-guide/) -- money market fund holdings and yields Back to top --- # Fund Shareholder Reports (N-CSR) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/fundshareholderreport-data-object-guide/#fund-shareholder-reports-n-csr-n-csrs-parse-certified-shareholder-reports-with-python) Fund Shareholder Reports (N-CSR / N-CSRS): Parse Certified Shareholder Reports with Python ========================================================================================== Registered investment companies file Form N-CSR annually and Form N-CSRS semiannually -- certified shareholder reports that include expense ratios, annual performance data, and top portfolio holdings for every share class. EdgarTools parses these Inline XBRL filings into structured Python objects using the OEF (Open-End Fund) taxonomy. `from edgar import get_filings filings = get_filings(form="N-CSR") report = filings[0].obj() report` ![Fund Shareholder Report N-CSR parsed with Python edgartools](https://edgartools.readthedocs.io/en/stable/images/fundshareholderreport-display.webp) Three lines to get a fully parsed certified shareholder report with fund name, net assets, expense ratios, and annual returns for every share class. * * * Access Expense Data ------------------- The `expense_data()` method returns a DataFrame with one row per share class, showing expense ratios and fees paid during the period: `report.expense_data()` | Column | What it is | | --- | --- | | `class_name` | Share class name (e.g., `"Investor Shares"`, `"Admiral Shares"`) | | `ticker` | Share class ticker symbol (if available) | | `expense_ratio_pct` | Total expense ratio as a decimal fraction | | `expenses_paid` | Total expenses paid in dollars during the period | | `advisory_fees_paid` | Advisory fees paid in dollars during the period | Expense ratios are stored as decimal fractions. An `expense_ratio_pct` of `0.0094` means a 0.94% expense ratio. Multiply by 100 to display as a percentage. * * * Access Performance Data ----------------------- The `performance_data()` method returns a DataFrame of average annual returns across all share classes and reporting periods: `report.performance_data()` | Column | What it is | | --- | --- | | `class_name` | Share class name | | `ticker` | Share class ticker symbol | | `period` | Return period label (e.g., `"1-Year"`, `"5-Year"`, `"10-Year"`) | | `return_pct` | Average annual return as a decimal fraction | | `inception_date` | Share class inception date (if reported) | Returns are decimal fractions. A `return_pct` of `0.1615` means a 16.15% return. Use this DataFrame to compare performance across share classes and time horizons in a single query. * * * Access Holdings Data -------------------- The `holdings_data()` method returns a DataFrame of top portfolio holdings disclosed in the report: `report.holdings_data()` | Column | What it is | | --- | --- | | `class_name` | Share class name | | `holding` | Holding name derived from the XBRL dimension member | | `pct_of_nav` | Holding as a percentage of net asset value (decimal fraction) | | `pct_of_total_inv` | Holding as a percentage of total investments (decimal fraction) | Holdings data availability varies significantly across filers. Some funds report detailed top-ten holdings; others omit this section entirely. Check `df.empty` before analysis. * * * Look Up a Specific Fund ----------------------- Search by management company name or CIK: `from edgar import Company company = Company("VANGUARD") filing = company.get_filings(form="N-CSR").latest(1) report = filing.obj() print(report.fund_name) # Fund name from the OEF taxonomy print(report.report_type) # "Annual" print(report.num_share_classes) # Number of share classes parsed print(report.net_assets) # Net assets (Decimal, or None)` A single N-CSR filing may cover multiple fund series within a complex. The `fund_name` property returns the first fund name found in the Inline XBRL document. * * * Access Semiannual Reports ------------------------- N-CSRS filings follow the same structure as N-CSR but cover the fund's semiannual period. The `is_annual` property distinguishes them: `from edgar import get_filings filings = get_filings(form="N-CSRS") report = filings[0].obj() print(report.report_type) # "Semi-Annual" print(report.is_annual) # False` Both form types return the same `FundShareholderReport` object. All three DataFrame methods work identically for both annual and semiannual reports. * * * Common Analysis Patterns ------------------------ ### Compare expense ratios across share classes `expenses = report.expense_data() # Display expense ratios as percentages expenses["expense_ratio_pct_display"] = expenses["expense_ratio_pct"] * 100 print(expenses[["class_name", "ticker", "expense_ratio_pct_display"]].to_string(index=False))` ### Sort share classes by long-term return `perf = report.performance_data() # Focus on 10-year returns ten_year = perf[perf["period"].str.contains("10", na=False)].copy() ten_year["return_display"] = ten_year["return_pct"] * 100 ten_year_sorted = ten_year.sort_values("return_display", ascending=False) print(ten_year_sorted[["class_name", "ticker", "return_display"]])` ### Check portfolio turnover `if report.portfolio_turnover is not None: turnover_pct = float(report.portfolio_turnover) * 100 print(f"Portfolio turnover: {turnover_pct:.1f}%") else: print("Portfolio turnover not reported")` ### Identify missing data before analysis `expenses = report.expense_data() performance = report.performance_data() holdings = report.holdings_data() for label, df in [("Expenses", expenses), ("Performance", performance), ("Holdings", holdings)]: if df.empty: print(f"{label}: not available in this filing") else: print(f"{label}: {len(df)} rows")` * * * Access Individual Objects ------------------------- The `share_classes` list provides direct access to each share class's `ShareClassInfo` object. These are useful for custom display, export, or filtering logic. ### Iterate over share classes `for sc in report.share_classes: ticker = f" ({sc.class_ticker})" if sc.class_ticker else "" print(f"{sc.class_name}{ticker}") print(f" Expense ratio: {float(sc.expense_ratio_pct) * 100:.2f}%" if sc.expense_ratio_pct else " Expense ratio: N/A") print(f" Holdings reported: {sc.holdings_count}" if sc.holdings_count else " Holdings count: N/A")` ### Inspect annual returns for a single class `sc = report.share_classes[0] for ret in sc.annual_returns: if ret.return_pct is not None: print(f" {ret.period_label}: {float(ret.return_pct) * 100:.2f}%")` ### Inspect top holdings for a single class `sc = report.share_classes[0] for holding in sc.holdings: if holding.pct_of_nav is not None: print(f" {holding.name}: {float(holding.pct_of_nav) * 100:.2f}% of NAV")` * * * Metadata Quick Reference ------------------------ | Property | Returns | Example | | --- | --- | --- | | `fund_name` | Fund name from OEF taxonomy | `"Vanguard 500 Index Fund"` | | `report_type` | `"Annual"` or `"Semi-Annual"` | `"Annual"` | | `is_annual` | Whether this is an N-CSR (annual) report | `True` | | `net_assets` | Net assets as `Decimal` (or `None`) | `Decimal("47382956000")` | | `portfolio_turnover` | Turnover rate as decimal fraction (or `None`) | `Decimal("0.7567")` | | `num_share_classes` | Number of share classes parsed | `3` | | `share_classes` | `List[ShareClassInfo]` for all share classes | Full per-class data | | `filing` | Source Filing object | `Filing` or `None` | | `cik` | CIK of the filer | `"0000102909"` | | `series_id` | SEC series ID | `"S000002277"` or `None` | * * * Methods Quick Reference ----------------------- | Method | Returns | What it does | | --- | --- | --- | | `expense_data()` | `DataFrame` | Expense ratios and fees for all share classes | | `performance_data()` | `DataFrame` | Average annual returns across all share classes and periods | | `holdings_data()` | `DataFrame` | Top holdings by percentage of NAV for all share classes | * * * Things to Know -------------- **Values are decimal fractions, not percentages.** Expense ratios, portfolio turnover, and return values are all stored as decimals. An `expense_ratio_pct` of `0.0094` means 0.94%. Multiply by 100 before displaying to users. **Net assets are in full dollars.** A `net_assets` value of `47382956000` means exactly $47.4 billion. There is no thousands scaling. **Annual and semiannual reports use the same object.** N-CSR (annual) and N-CSRS (semiannual) filings both produce a `FundShareholderReport`. Use `is_annual` to distinguish them. **XBRL is required.** `from_filing()` calls `filing.xbrl()` internally. If the filing does not contain Inline XBRL -- which is uncommon but possible for older filings -- `filing.obj()` returns `None`. Always check for `None` before accessing properties. **Ticker symbols are often absent.** The OEF taxonomy includes a `ClassTicker` concept, but many filers do not populate it. The `class_ticker` field will be `None` for most share classes from smaller fund complexes. **Holdings by NAV are sparsely populated.** The `pct_of_nav` field requires the filer to tag `oef:HoldingPctOfNav` against a `HoldingAxis` dimension. Many filers report only `holdings_count` (the total number of holdings) without disclosing individual holding percentages. **Multiple fund series per filing.** A single N-CSR filing can cover multiple series from a fund family. Only the first `oef:FundName` fact is captured in `fund_name`. The DataFrame methods aggregate data across all discovered share classes regardless of series. **Share class discovery uses ClassAxis dimensions.** Share classes are identified from the `oef:ClassAxis` dimension in the Inline XBRL. For single-class funds that do not use this dimension, a single placeholder class is constructed from undimensioned facts. **Approximately 6,600 filings per year.** N-CSR and N-CSRS together account for roughly 6,623 annual filings from registered investment companies, covering thousands of individual fund series. * * * Related ------- * [Fund Entities](https://edgartools.readthedocs.io/en/stable/guides/fund-entity-guide/) -- look up funds by ticker, navigate hierarchies * [Working with Filings](https://edgartools.readthedocs.io/en/stable/guides/working-with-filing/) -- general filing access patterns * [Fund Portfolios (N-PORT)](https://edgartools.readthedocs.io/en/stable/guides/nport-data-object-guide/) -- monthly fund portfolio holdings * [Money Market Funds (N-MFP)](https://edgartools.readthedocs.io/en/stable/guides/moneymarketfund-data-object-guide/) -- money market fund holdings and yields * [Fund Census (N-CEN)](https://edgartools.readthedocs.io/en/stable/guides/fundcensus-data-object-guide/) -- annual fund operational census with service provider data Back to top --- # Fund Portfolios (N-PORT) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/nport-data-object-guide/#n-port-parse-mutual-fund-portfolio-holdings-with-python) N-PORT: Parse Mutual Fund Portfolio Holdings with Python ======================================================== Every registered mutual fund and ETF files Form NPORT-P with the SEC each month, disclosing their complete portfolio -- every stock, bond, swap, and option they hold. EdgarTools parses these filings into structured Python objects so you can analyze fund portfolios in a few lines of code. `from edgar import get_filings filings = get_filings(form="NPORT-P") report = filings[0].obj() report` ![N-PORT mutual fund portfolio holdings parsed with Python edgartools](https://edgartools.readthedocs.io/en/latest/images/nport-p.webp) Three lines to get a fully parsed fund report with total assets, all investment positions, derivative exposures, and risk metrics. * * * Investment Data --------------- The `investment_data()` method returns a DataFrame with one row per position, sorted by absolute value: `report.investment_data()` | Column | What it is | | --- | --- | | `name` | Issuer name (`"APPLE INC"`) | | `title` | Security title (`"COMMON STOCK"`) | | `cusip` | 9-character CUSIP | | `ticker` | Resolved ticker symbol | | `balance` | Share count or par amount | | `units` | `"NS"` (shares), `"PA"` (principal), or `"NC"` (contracts) | | `value_usd` | Market value in **US dollars** | | `pct_value` | Percentage of net asset value | | `asset_category` | SEC category (`"EC"` equity, `"DBT"` debt, etc.) | | `issuer_category` | Issuer type (`"CORP"`, `"USG"`, `"MUN"`, etc.) | | `currency_code` | Currency of the position | | `investment_country` | Country code | | `is_derivative` | `True` for derivatives, `False` for securities | Values are in full US dollars -- unlike 13F filings, NPORT values are **not** in thousands. * * * Securities vs Derivatives ------------------------- Split the portfolio into traditional securities and derivative positions: `# Non-derivative positions only (stocks, bonds, etc.) report.securities_data() # Derivative positions only report.derivatives_data()` The `derivatives_data()` DataFrame adds columns specific to derivatives: | Column | What it is | | --- | --- | | `subtype` | Derivative subtype (e.g., `"CDS"`, `"IRS"`) | | `reference` | Reference entity or index | | `counterparty` | Counterparty name | | `notional_amount` | Notional value | | `unrealized_pnl` | Unrealized appreciation/depreciation | | `termination_date` | Contract expiration date | | `pct_value` | Percentage of NAV | See it live on edgar.tools The code above parses N-PORT portfolio holdings in Python. **edgar.tools** renders fund portfolios visually — browse holdings, derivative exposures, and risk metrics for any registered fund. * **[Browse company filings and fund data →](https://app.edgar.tools/companies?utm_source=edgartools-docs&utm_medium=see-live&utm_content=nport-guide) ** * **[Watch filings arrive in real time →](https://app.edgar.tools/filings?utm_source=edgartools-docs&utm_medium=see-live&utm_content=nport-guide) ** Free tier available. Also includes a REST API for programmatic access. [Pricing →](https://app.edgar.tools/pricing?utm_source=edgartools-docs&utm_medium=see-live&utm_content=nport-guide) * * * Derivative-Specific DataFrames ------------------------------ For detailed analysis of specific derivative types, use the dedicated methods. Each returns a DataFrame with type-specific fields: `report.swaps_data() # Interest rate swaps, CDS, total return swaps report.options_data() # Options with strike price, expiry, put/call report.forwards_data() # FX forwards with currencies and settlement report.futures_data() # Futures with payoff profile and expiry report.swaptions_data() # Swaption contracts` All derivative DataFrames share common fields (`name`, `title`, `counterparty`, `notional_amount`, `unrealized_pnl`) plus type-specific columns. For example, `swaps_data()` includes directional receive/pay legs, and `options_data()` includes `strike_price`, `written_or_purchased`, and `shares_per_contract`. * * * Fund Financials --------------- Access the fund's balance sheet data through `fund_info`: `print(f"Total assets: ${report.fund_info.total_assets:,.0f}") print(f"Total liabilities: ${report.fund_info.total_liabilities:,.0f}") print(f"Net assets: ${report.fund_info.net_assets:,.0f}")` * * * Risk Metrics ------------ ### Interest rate sensitivity NPORT filings include DV01 (dollar value of a 1 basis point move) and DV100 (dollar value of a 100 basis point move) across multiple time horizons: `for currency, metric in report.fund_info.current_metrics.items(): print(f"{currency} DV01 (1yr): {metric.intrstRtRiskdv01.period1Yr}") print(f"{currency} DV100 (1yr): {metric.intrstRtRiskdv100.period1Yr}")` ### Credit spread risk Investment grade and non-investment grade spread sensitivity, when reported: `ig = report.fund_info.credit_spread_risk_investment_grade if ig: print(f"IG spread risk (1yr): {ig.period1Yr}") nig = report.fund_info.credit_spread_risk_non_investment_grade if nig: print(f"Non-IG spread risk (1yr): {nig.period1Yr}")` * * * Look Up a Specific Fund ----------------------- Search for a fund by its CIK or by the management company: `from edgar import Company # By management company CIK vanguard = Company("0000102909") filing = vanguard.get_filings(form="NPORT-P").latest(1) report = filing.obj() print(report.general_info.name) # Management company print(report.general_info.series_name) # Specific fund/series print(report.reporting_period) # Report date` Or use the `Fund` class for a simpler path: `from edgar import Fund fund = Fund("VFINX") report = fund.get_latest_report() # Latest NPORT-P report df = fund.get_portfolio() # Portfolio as DataFrame` * * * Common Analysis Patterns ------------------------ ### Top holdings by value `df = report.investment_data(include_derivatives=False) total = df['value_usd'].sum() df['weight'] = (df['value_usd'] / total * 100).round(2) df[['name', 'ticker', 'value_usd', 'weight']].head(10)` ### Sector allocation `df = report.securities_data() df.groupby('asset_category')['value_usd'].sum().sort_values(ascending=False)` ### Derivative exposure as percentage of NAV `net_assets = report.fund_info.net_assets deriv = report.derivatives_data() if not deriv.empty: total_notional = deriv['notional_amount'].abs().sum() print(f"Derivative notional: ${total_notional:,.0f}") print(f"As % of NAV: {total_notional / net_assets * 100:.1f}%")` * * * Metadata Quick Reference ------------------------ | Property | Returns | Example | | --- | --- | --- | | `general_info.name` | Management company | `"VANGUARD CHESTER FUNDS"` | | `general_info.series_name` | Fund/series name | `"Vanguard Target Retirement 2035 Fund"` | | `general_info.cik` | SEC CIK | `"0000102909"` | | `general_info.rep_period_date` | Report date | `"2024-03-31"` | | `general_info.fiscal_year_end` | Fiscal year end | `"0131"` | | `general_info.series_id` | SEC series ID | `"S000004104"` | | `general_info.reg_lei` | LEI | `"549300..."` | | `reporting_period` | Same as rep\_period\_date | `"2024-03-31"` | | `name` | Company - Series | `"VANGUARD CHESTER FUNDS - Vanguard Target..."` | | `has_investments` | Has any positions? | `True` | | `header.submission_type` | Form type filed | `"NPORT-P"` | | `filing` | Source Filing object | `Filing` or `None` | | `cik` | CIK of the filer | `"0000102909"` | | `series_id` | SEC series ID | `"S000004104"` or `None` | * * * Methods Quick Reference ----------------------- | Call | Returns | What it does | | --- | --- | --- | | `report.investment_data()` | `DataFrame` | All positions (securities + derivatives) | | `report.securities_data()` | `DataFrame` | Non-derivative positions only | | `report.derivatives_data()` | `DataFrame` | Derivative positions with P&L | | `report.swaps_data()` | `DataFrame` | Swap details with receive/pay legs | | `report.options_data()` | `DataFrame` | Options with strike, expiry, put/call | | `report.forwards_data()` | `DataFrame` | FX forwards with settlement info | | `report.futures_data()` | `DataFrame` | Futures contracts | | `report.swaptions_data()` | `DataFrame` | Swaption contracts | | `report.derivatives` | `list` | Raw derivative investment objects | | `report.non_derivatives` | `list` | Raw non-derivative investment objects | | `report.get_fund_series()` | `FundSeries` | Fund series object | | `report.get_ticker_for_series()` | `str` | Ticker for this series | | `report.get_tickers_for_series()` | `list[str]` | All tickers for this series | | `report.matches_ticker(ticker)` | `bool` | Whether report matches a ticker | * * * Things to Know -------------- **Values are in full dollars.** Unlike 13F filings (which report in thousands), NPORT values are in actual USD. A `value_usd` of 135,364,000 means exactly $135.4 million. **NPORT-P vs NPORT-EX.** NPORT-P is the full monthly filing with all positions and risk metrics. NPORT-EX is a quarterly exhibit containing only a subset of holdings. EdgarTools handles both, but NPORT-P has the most complete data. **Monthly reporting.** Funds file NPORT-P every month, but only the quarter-end filings are made public. The other two months in each quarter are confidential (released after 60 days). **Derivative coverage.** EdgarTools parses all five derivative types: swaps, options, forwards, futures, and swaptions. Each type has a dedicated `*_data()` method for detailed analysis. **Ticker resolution.** Tickers are resolved from CUSIPs and fund metadata. Most common securities resolve correctly, but obscure or private placements may have blank tickers. **Empty DataFrames.** Methods like `derivatives_data()` and `swaps_data()` return an empty DataFrame when no positions of that type exist. Always check with `df.empty` before analysis. * * * Related ------- * [Fund Entities](https://edgartools.readthedocs.io/en/latest/guides/fund-entity-guide/) -- look up funds by ticker, navigate hierarchies * [Working with Filings](https://edgartools.readthedocs.io/en/latest/guides/working-with-filing/) -- general filing access patterns Back to top --- # Fund Census (N-CEN) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/fundcensus-data-object-guide/#fund-census-n-cen-parse-annual-fund-reports-with-python) Fund Census (N-CEN): Parse Annual Fund Reports with Python ========================================================== Every registered investment company files Form N-CEN annually -- a comprehensive census of fund operational data. EdgarTools parses these filings into structured Python objects, revealing fund series, service providers, board composition, ETF mechanics, broker relationships, and securities lending programs. `from edgar import get_filings filings = get_filings(form="N-CEN") census = filings[0].obj() census` ![Fund Census N-CEN report parsed with Python edgartools](https://edgartools.readthedocs.io/en/stable/images/fundcensus-display.webp) Three lines to get a fully parsed annual census with registrant info, fund series, directors, and all operational disclosures. * * * Access Series Data ------------------ The `series_data()` method returns a DataFrame with one row per fund series, showing assets, commission totals, and service provider counts: `census.series_data()` | Column | What it is | | --- | --- | | `name` | Fund series name | | `series_id` | SEC series identifier (e.g., `"S000052123"`) | | `lei` | Legal Entity Identifier | | `fund_type` | Fund classification | | `avg_net_assets` | Average monthly net assets | | `aggregate_commission` | Total broker commissions paid | | `num_advisers` | Count of investment advisers | | `num_custodians` | Count of custodians | | `has_etf` | Whether series has ETF structure | Average net assets and commission figures are in full dollars (not thousands). An `avg_net_assets` of 19,574,292 means exactly $19.6 million. * * * Access Service Provider Network ------------------------------- See who manages, custodies, administers, and services the funds: `census.service_providers()` Returns a DataFrame with all service providers flattened across fund series: | Column | What it is | | --- | --- | | `series_name` | Fund series name | | `series_id` | Series identifier | | `role` | Provider role (`"adviser"`, `"custodian"`, `"transfer agent"`, etc.) | | `provider_name` | Provider firm name | | `lei` | Legal Entity Identifier | | `affiliated` | Whether provider is affiliated with the fund | This includes advisers, custodians, transfer agents, administrators, pricing services, and shareholder servicing agents -- everyone touching the fund's operations. * * * Analyze Broker Relationships ---------------------------- N-CEN discloses broker-dealer and broker commission payments: `census.broker_data()` | Column | What it is | | --- | --- | | `series_name` | Fund series name | | `series_id` | Series identifier | | `type` | `"broker-dealer"` or `"broker"` | | `broker_name` | Firm name | | `lei` | Legal Entity Identifier | | `commission` | Commission amount paid | Commission values are in full dollars. Use this to analyze broker concentration and trading costs across the fund complex. * * * Examine Board Composition ------------------------- Access the board of directors with interested-person flags: `census.director_data()` | Column | What it is | | --- | --- | | `name` | Director name | | `crd_number` | CRD identifier (if registered) | | `interested_person` | Whether director is an interested person under the '40 Act | Directors marked as "interested persons" have material relationships with the fund or its adviser. Independent directors are not interested persons. * * * Access ETF-Specific Data ------------------------ For funds with ETF structures, get creation/redemption mechanics and authorized participants: `census.etf_data()` Returns a DataFrame with ETF-specific metrics: | Column | What it is | | --- | --- | | `series_name` | Fund series name | | `series_id` | Series identifier | | `exchange` | Primary exchange (e.g., `"NYSE"`, `"NASDAQ"`) | | `ticker` | Trading symbol | | `creation_unit_size` | Number of shares per creation unit | | `avg_pct_purchased_in_kind` | Average percentage of creations settled in-kind | | `avg_pct_redeemed_in_kind` | Average percentage of redemptions settled in-kind | | `is_in_kind` | Whether fund permits in-kind transactions | | `num_authorized_participants` | Count of authorized participants | This DataFrame is empty for fund complexes without ETFs. Always check with `df.empty` before analysis. * * * Look Up a Specific Fund Complex ------------------------------- Search by management company CIK or name: `from edgar import Company company = Company("VANGUARD") filing = company.get_filings(form="N-CEN").latest(1) census = filing.obj() print(census.name) # Registrant name print(f"{census.num_series} series") # Series count in this filing print(census.report_date) # Report period end` A single N-CEN filing may cover multiple fund series, or just one series from a larger complex. Check `census.total_series` to see how many series the registrant has total. * * * Common Analysis Patterns ------------------------ ### Service provider concentration `providers = census.service_providers() # Count unique providers per role providers.groupby('role')['provider_name'].nunique() # Find affiliated providers affiliated = providers[providers['affiliated'] == True]` ### Commission analysis `brokers = census.broker_data() if not brokers.empty: # Top brokers by commission top_brokers = brokers.groupby('broker_name')['commission'].sum().sort_values(ascending=False) print(top_brokers.head(10))` ### Board independence `directors = census.director_data() total = len(directors) interested = directors['interested_person'].sum() independent = total - interested print(f"Board composition: {independent} independent, {interested} interested")` ### ETF creation mechanics `etf = census.etf_data() if not etf.empty: for idx, row in etf.iterrows(): print(f"{row['ticker']}: {row['avg_pct_purchased_in_kind']:.1f}% in-kind purchases")` * * * Access Individual Objects ------------------------- The FundCensus class uses Pydantic models for structured access to all fund census data. These are useful when building custom applications or exporting to other formats. ### Registrant Information `reg = census.registrant print(reg.name) # Registrant name print(reg.cik) # CIK print(reg.classification_type) # Fund classification print(reg.total_series) # Total series count print(reg.accountant.name) # Public accountant print(reg.cco_name) # Chief Compliance Officer` ### Fund Series `for series in census.series: print(f"{series.name} ({series.series_id})") print(f" Advisers: {len(series.advisers)}") print(f" Custodians: {len(series.custodians)}") print(f" Securities lending: {series.is_securities_lending}") # ETF details if series.etf_info: etf = series.etf_info print(f" ETF ticker: {etf.ticker}") print(f" Exchange: {etf.exchange}") print(f" Creation unit: {etf.creation_unit_size}")` ### Service Providers `series = census.series[0] # Investment advisers for adviser in series.advisers: print(f"{adviser.name} - {adviser.role}") print(f" Affiliated: {adviser.is_affiliated}") # Custodians for custodian in series.custodians: print(f"{custodian.name}")` ### Broker Dealers `series = census.series[0] for bd in series.broker_dealers: print(f"{bd.name}") print(f" Commission: ${bd.commission:,.2f}" if bd.commission else " No commission reported") print(f" LEI: {bd.lei}")` ### Line of Credit `series = census.series[0] if series.line_of_credit: loc = series.line_of_credit if loc.has_line_of_credit: print(f"Line of credit: ${loc.size:,.0f}") print(f"Committed: {loc.is_committed}") print(f"Institutions: {', '.join(loc.institution_names)}")` ### Securities Lending `series = census.series[0] for sl in series.securities_lending: print(f"Agent: {sl.agent_name}") print(f" Affiliated: {sl.is_affiliated}") print(f" Indemnified: {sl.is_indemnified}")` * * * Metadata Quick Reference ------------------------ | Property | Returns | Example | | --- | --- | --- | | `name` | Registrant name | `"360 Funds"` | | `cik` | CIK identifier | `"0001319067"` | | `report_date` | Report period end date | `"2025-11-30"` | | `num_series` | Number of series in filing | `1` | | `total_series` | Total series registered | `8` | | `classification_type` | Fund classification | `"N-1A"` | | `is_etf_company` | Has any ETF series | `True` or `False` | | `registrant` | RegistrantInfo object | Full registrant details | | `series` | List of FundSeriesInfo | All series with full details | | `filing` | Source Filing object | `Filing` or `None` | * * * Methods Quick Reference ----------------------- | Method | Returns | What it does | | --- | --- | --- | | `series_data()` | `DataFrame` | Fund series summary with assets and counts | | `service_providers()` | `DataFrame` | All service providers flattened across series | | `broker_data()` | `DataFrame` | Broker-dealer and broker commission data | | `director_data()` | `DataFrame` | Board of directors with interested-person flags | | `etf_data()` | `DataFrame` | ETF-specific metrics for ETF series | * * * Things to Know -------------- **Values are in full dollars.** Unlike 13F filings (which report in thousands), N-CEN values are in actual USD. An `avg_net_assets` of 19,574,292 means exactly $19.6 million. **Annual filings.** N-CEN is filed once per year, unlike monthly N-PORT or N-MFP reports. Each filing covers the fund's fiscal year. **Multiple series.** A single N-CEN filing may cover one series or dozens, depending on how the fund complex structures its filings. Check `num_series` vs `total_series` to understand coverage. **N/A sentinel values.** The XML uses "N/A" as a sentinel. EdgarTools converts these to `None` automatically. **Boolean display.** Pydantic booleans render as "Yes/No" in the Rich display for readability. **Service provider diversity.** Funds use many service provider types: advisers, custodians, transfer agents, administrators, pricing services, shareholder servicing agents. All are captured in `service_providers()`. **ETF mechanics.** Only series with ETF structures have `etf_info` populated. Use `etf_data()` to filter to ETF series automatically. **Securities lending.** Not all funds engage in securities lending. Check `is_securities_lending` or the length of `series.securities_lending`. **Line of credit.** Funds may have committed or uncommitted lines of credit with one or more institutions. The `line_of_credit` object captures all details. **Director CRD numbers.** Only registered directors have CRD numbers. Many directors won't have one -- this is normal. * * * Related ------- * [Fund Entities](https://edgartools.readthedocs.io/en/stable/guides/fund-entity-guide/) -- look up funds by ticker, navigate hierarchies * [Working with Filings](https://edgartools.readthedocs.io/en/stable/guides/working-with-filing/) -- general filing access patterns * [Fund Portfolios (N-PORT)](https://edgartools.readthedocs.io/en/stable/guides/nport-data-object-guide/) -- monthly fund portfolio holdings * [Money Market Funds (N-MFP)](https://edgartools.readthedocs.io/en/stable/guides/moneymarketfund-data-object-guide/) -- money market fund holdings and yields Back to top --- # Sale Notices (Form 144) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/form144-data-object-guide/#form-144-parse-insider-restricted-stock-sale-notices) Form 144: Parse Insider Restricted Stock Sale Notices ===================================================== Form 144 is a notice of proposed sale of restricted securities under SEC Rule 144. Filed by company insiders (officers, directors, 10%+ shareholders) before selling restricted or control securities. This guide details all data available from the `Form144` class for building views. * * * Overview -------- | Property | Type | Description | | --- | --- | --- | | Class Name | `Form144` | | | Forms Handled | `144`, `144/A` | | | Module | `edgar.form144` | | | Source Data | XML document | | * * * Basic Metadata -------------- | Property | Type | Description | Example | | --- | --- | --- | --- | | `filing_date` | `str` | Date filed with SEC | `"2023-04-18"` | | `is_amendment` | `bool` | Whether this is a 144/A amendment | `False` | * * * Seller Information ------------------ | Property | Type | Description | Example | | --- | --- | --- | --- | | `person_selling` | `str` | Name of person selling | `"John Smith"` | | `relationships` | `List[str]` | Relationship(s) to issuer | `["Director", "Officer"]` | | `address` | `Address` | Seller's address | See Address object | | `filer` | `Filer` | SEC filer credentials | CIK, entity name, file number | | `contact` | `Contact` | Contact information | Name, phone, email | ### Address Object | Property | Type | Description | | --- | --- | --- | | `street1` | `str` | Street line 1 | | `street2` | `str` | Street line 2 | | `city` | `str` | City | | `state_or_country` | `str` | State/country code | | `zipcode` | `str` | ZIP code | * * * Issuer Information ------------------ | Property | Type | Description | Example | | --- | --- | --- | --- | | `issuer_name` | `str` | Company name | `"Owens Corning"` | | `issuer_cik` | `str` | Company CIK | `"1370946"` | | `sec_file_number` | `str` | SEC file number | `"001-36390"` | | `issuer_contact_phone` | `str` | Issuer contact phone | `"419-248-8000"` | | `company` | `Company` | Full Company object | Lazy-loaded from CIK | * * * Securities Information (Proposed Sale) -------------------------------------- The core data about the securities to be sold. ### Quick Access Properties (Single Security) For filings with one security type (most common): | Property | Type | Description | Example | | --- | --- | --- | --- | | `units_to_be_sold` | `int` | Total shares to sell | `3000` | | `market_value` | `float` | Total market value | `300000.0` | | `security_class` | `str` | Security type | `"Common"` | | `approx_sale_date` | `str` | Approximate sale date | `"04/18/2023"` | | `exchange_name` | `str` | Exchange for sale | `"NYSE"` | | `broker_name` | `str` | Broker executing sale | `"Fidelity Brokerage Services LLC"` | ### Aggregation Properties (Multi-Security) For filings with multiple security types: | Property | Type | Description | | --- | --- | --- | | `total_units_to_be_sold` | `int` | Sum across all securities | | `total_market_value` | `float` | Sum across all securities | | `num_securities` | `int` | Count of security types | | `is_multi_security` | `bool` | Whether multiple security types | ### Securities Information Holder Access the full DataFrame via `securities_info`: `form144 = filing.obj() # Holder object with aggregation methods holder = form144.securities_info # Properties holder.empty # bool - is data empty? holder.total_units_to_be_sold # int - sum of all units holder.total_market_value # float - sum of market values holder.security_classes # List[str] - all security types holder.exchanges # List[str] - unique exchanges holder.brokers # List[str] - unique brokers holder.percent_of_outstanding # float - % of outstanding shares holder.avg_price_per_unit # float - market_value / units # Iteration for row in holder: print(row.security_class, row.units_to_be_sold) # Raw DataFrame df = form144.securities_information # pd.DataFrame` ### Securities Information DataFrame Columns | Column | Type | Description | | --- | --- | --- | | `security_class` | `str` | Security type (Common, Preferred, etc.) | | `units_to_be_sold` | `int` | Number of units | | `market_value` | `float` | Aggregate market value | | `units_outstanding` | `int` | Total outstanding shares | | `approx_sale_date` | `str` | Expected sale date (MM/DD/YYYY) | | `exchange_name` | `str` | Exchange code (NYSE, NASDAQ, etc.) | | `broker_name` | `str` | Broker/market maker name | * * * Securities To Be Sold (Acquisition History) ------------------------------------------- Details about how/when the seller acquired the securities. ### Access `# Holder with aggregation holder = form144.securities_selling holder.empty # bool holder.total_amount_acquired # int - sum of all acquired holder.acquisition_dates # List[str] - unique dates holder.has_gift_transactions # bool - any gifts? # Iteration for row in holder: print(row.acquired_date, row.amount_acquired) # Raw DataFrame df = form144.securities_to_be_sold` ### Securities To Be Sold DataFrame Columns | Column | Type | Description | | --- | --- | --- | | `security_class` | `str` | Security type | | `acquired_date` | `str` | Date acquired (MM/DD/YYYY) | | `amount_acquired` | `int` | Shares acquired | | `nature_of_acquisition` | `str` | How acquired (e.g., "Employee Stock Award") | | `acquired_from` | `str` | Person/entity acquired from | | `is_gift` | `str` | Gift transaction? ("Y"/"N") | | `donar_acquired_date` | `str` | If gift, when donor acquired | | `payment_date` | `str` | Date payment made | | `nature_of_payment` | `str` | Payment type (e.g., "CASH") | * * * Securities Sold Past 3 Months ----------------------------- Prior sales within the last 90 days (SEC Rule 144 volume limits). ### Access `# Holder with aggregation holder = form144.recent_sales holder.empty # bool holder.total_amount_sold # int - sum of amounts holder.total_gross_proceeds # float - sum of proceeds holder.sellers # List[str] - unique seller names # Iteration for row in holder: print(row.sale_date, row.amount_sold, row.gross_proceeds) # Raw DataFrame df = form144.securities_sold_past_3_months # Flag for no activity form144.nothing_to_report # bool - no sales in past 3 months` ### Aggregation Properties | Property | Type | Description | | --- | --- | --- | | `total_amount_sold_past_3_months` | `int` | Total shares sold recently | | `total_gross_proceeds_past_3_months` | `float` | Total proceeds | ### Securities Sold Past 3 Months DataFrame Columns | Column | Type | Description | | --- | --- | --- | | `security_class` | `str` | Security type | | `seller_name` | `str` | Seller name | | `sale_date` | `str` | Date of sale (MM/DD/YYYY) | | `amount_sold` | `int` | Shares sold | | `gross_proceeds` | `float` | Sale proceeds | * * * Analyst Metrics (Computed Properties) ------------------------------------- Pre-computed metrics for investment analysis: ### Percentage Metrics | Property | Type | Description | | --- | --- | --- | | `percent_of_holdings` | `float` | % of outstanding shares being sold | | `avg_price_per_unit` | `float` | Market value / units to sell | ### Holding Period Analysis | Property | Type | Description | | --- | --- | --- | | `holding_period_days` | `int` or `None` | Avg days held before sale | | `holding_period_years` | `float` or `None` | Avg years held | ### 10b5-1 Plan Compliance | Property | Type | Description | | --- | --- | --- | | `is_10b5_1_plan` | `bool` | Sale under 10b5-1 trading plan | | `days_since_plan_adoption` | `int` or `None` | Days from plan adoption to sale | | `cooling_off_compliant` | `bool` or `None` | 90-day cooling off observed (post-2022 rule) | ### Anomaly Detection Flags | Property | Type | Description | | --- | --- | --- | | `is_large_liquidation` | `bool` | Selling >5% of outstanding | | `is_short_hold` | `bool` | Holding period <1 year | | `has_multiple_plans` | `bool` | Multiple 10b5-1 plan dates | | `anomaly_flags` | `List[str]` | All triggered flags | **Possible Anomaly Flags:** - `LARGE_LIQUIDATION` - Selling >5% of outstanding shares - `SHORT_HOLD` - Held <1 year before selling - `COOLING_OFF_VIOLATION` - <90 days since 10b5-1 plan adoption - `MULTIPLE_PLANS` - Multiple 10b5-1 plan adoption dates * * * Notice Signature ---------------- | Property | Type | Description | | --- | --- | --- | | `notice_signature.notice_date` | `str` | Date notice signed | | `notice_signature.signature` | `str` | Signature text | | `notice_signature.plan_adoption_dates` | `List[str]` | 10b5-1 plan dates | * * * Remarks ------- | Property | Type | Description | | --- | --- | --- | | `remarks` | `str` | Additional remarks from filer | * * * Summary Methods --------------- ### get\_summary() Returns dict with key filing info: `summary = form144.get_summary() # Returns: { 'person_selling': str, 'issuer': str, 'issuer_cik': str, 'relationships': List[str], 'num_securities': int, 'total_units_to_be_sold': int, 'total_market_value': float, 'security_classes': List[str], 'exchanges': List[str], 'nothing_to_report_past_3_months': bool, 'total_sold_past_3_months': int, 'is_amendment': bool, 'filing_date': str, }` ### to\_analyst\_summary() Returns dict optimized for investment screening: `summary = form144.to_analyst_summary() # Returns: { # Identity 'person_selling': str, 'issuer': str, 'issuer_cik': str, 'relationships': List[str], 'filing_date': str, # Sale metrics 'units_to_sell': int, 'market_value': float, 'percent_of_holdings': float, 'avg_price_per_unit': float, # Timing 'sale_date': str, 'holding_period_years': float or None, # 10b5-1 Plan 'is_10b5_1_plan': bool, 'days_since_plan_adoption': int or None, 'cooling_off_compliant': bool or None, # Recent activity 'sold_past_3_months': int, 'proceeds_past_3_months': float, # Flags 'anomaly_flags': List[str], # Metadata 'is_amendment': bool, 'exchange': str, 'broker': str, }` ### to\_dataframe() Returns DataFrame with one row per security: `df = form144.to_dataframe() # Columns: security_class, units_to_be_sold, market_value, units_outstanding, # approx_sale_date, exchange_name, broker_name, person_selling, # issuer, issuer_cik, filing_date, is_amendment` * * * View Design Recommendations --------------------------- ### Primary View Components 1. **Header Section** 2. Issuer name + CIK 3. Form type (144 or 144/A amendment) 4. Filing date 5. Amendment indicator if applicable 6. **Seller Panel** 7. Person selling (prominent) 8. Relationship(s) to issuer (badges/tags) 9. Contact info (collapsible) 10. **Sale Summary Card** 11. Units to be sold (large, highlighted) 12. Market value 13. % of holdings 14. Avg price per unit 15. Approximate sale date 16. Exchange / Broker 17. **Compliance Panel** 18. 10b5-1 Plan: Yes/No indicator 19. Days since plan adoption 20. Cooling off status (green checkmark or red warning) 21. Anomaly flags (warning badges) 22. **Securities Information Table** 23. Security class, units, market value, sale date 24. Exchange, broker for each 25. **Acquisition History Table** (collapsible) 26. How/when securities were acquired 27. Gift transaction indicators 28. **Recent Sales Table** (collapsible) 29. Past 3 months activity 30. Or "Nothing to report" message 31. **Signature Footer** 32. Notice date 33. Signature 34. Plan adoption dates ### Data Priority for Display | Priority | Data | Reason | | --- | --- | --- | | High | Person selling + relationships | Who is selling | | High | Units, market value, % holdings | Sale magnitude | | High | Anomaly flags | Risk indicators | | Medium | 10b5-1 compliance | Regulatory context | | Medium | Sale date, exchange, broker | Transaction details | | Medium | Recent sales (past 3 months) | Volume limit context | | Low | Acquisition history | Background detail | | Low | Full address, contact | Administrative | ### Visual Indicators (Suggested) | Condition | Visual Treatment | | --- | --- | | `is_amendment` | Yellow "Amendment" badge | | `is_large_liquidation` | Red warning icon | | `is_short_hold` | Orange warning icon | | `cooling_off_compliant == False` | Red "Violation" badge | | `is_10b5_1_plan` | Blue "10b5-1" badge | | Large `market_value` (>$1M) | Emphasized styling | ### Anomaly Flags Color Coding | Flag | Color | Meaning | | --- | --- | --- | | `LARGE_LIQUIDATION` | Red | \>5% of company being sold | | `SHORT_HOLD` | Orange | Held less than 1 year | | `COOLING_OFF_VIOLATION` | Red | 10b5-1 rule violation | | `MULTIPLE_PLANS` | Yellow | Unusual plan activity | * * * Example Data Structure ---------------------- `{ # Identity "person_selling": "Brian O. Chambers", "issuer_name": "Owens Corning", "issuer_cik": "1370946", "relationships": ["Officer"], "filing_date": "2023-04-18", "is_amendment": False, # Sale summary "units_to_be_sold": 3000, "market_value": 300000.0, "percent_of_holdings": 0.0028, "avg_price_per_unit": 100.0, "security_class": "Common", "approx_sale_date": "04/18/2023", "exchange_name": "NYSE", "broker_name": "Fidelity Brokerage Services LLC", # Compliance "is_10b5_1_plan": True, "days_since_plan_adoption": 120, "cooling_off_compliant": True, "holding_period_years": 2.5, # Flags "anomaly_flags": [], # Recent activity "nothing_to_report": True, "total_sold_past_3_months": 0, "total_proceeds_past_3_months": 0.0, # Securities info (array) "securities_information": [ { "security_class": "Common", "units_to_be_sold": 3000, "market_value": 300000.0, "units_outstanding": 107000000, "approx_sale_date": "04/18/2023", "exchange_name": "NYSE", "broker_name": "Fidelity Brokerage Services LLC" } ], # Acquisition history (array) "securities_to_be_sold": [ { "security_class": "Common", "acquired_date": "03/15/2021", "amount_acquired": 500, "nature_of_acquisition": "Employee Stock Award", "acquired_from": "Issuer", "is_gift": "N" } // ... more entries ], # Signature "notice_date": "04/17/2023", "signature": "/s/ Brian O. Chambers", "plan_adoption_dates": ["01/15/2023"] }` * * * Notes for Implementation ------------------------ 1. **XML-Only**: Form 144 data comes exclusively from XML. Filings before ~2015 may not have XML and will return `None` from `filing.obj()`. 2. **Multiple Securities**: Some filings contain multiple security types. Always use aggregation properties (`total_*`) or iterate through `securities_info`. 3. **Placeholder Dates**: SEC forms use `01/01/1933` as placeholder dates. The class filters these out in computed metrics. 4. **Holding Period**: Calculated from acquisition dates to sale date. Returns `None` if dates are invalid or missing. 5. **Cooling Off Rule**: The 90-day requirement took effect in 2022. Earlier filings may show "violations" that weren't violations at the time. Back to top --- # ABS Distribution (10-D) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/tend-data-object-guide/#form-10-d-abs-distribution-reports-cmbs) Form 10-D: ABS Distribution Reports (CMBS) ========================================== Overview -------- **Form 10-D** is an Asset-Backed Issuer Distribution Report required under Sections 13 and 15(d) of the Securities Exchange Act of 1934. These filings disclose distribution and pool performance data for publicly offered asset-backed securities (ABS). ABS come in many forms — credit card receivables, auto loans, student loans, residential mortgages (RMBS), commercial mortgages (CMBS), and utility securitizations. However, **edgartools currently supports only CMBS (Commercial Mortgage-Backed Securities) filings** that include structured XML asset data in the EX-102 attachment. For CMBS 10-D filings, edgartools extracts: - **Loan-level data**: 49 fields covering origination, balance, rate, maturity, servicer, and payment status - **Property-level data**: 46 fields covering location, type, valuation, occupancy, financials, and debt service coverage ratios - **Summary statistics**: Pool metrics like total balance, average DSCR, occupancy, and property type distribution ### What About Non-CMBS 10-D Filings? For non-CMBS 10-D filings (auto loans, credit cards, student loans, etc.), `filing.obj()` returns `None` because these filings do not include standardized XML asset data. Asset data for non-CMBS filings is typically reported in separate ABS-EE filings or in HTML tables with highly variable formats. Access Pattern -------------- `from edgar import Filing # Get a Form 10-D filing filing = Filing(form="10-D", ...) # Parse into TenD object ten_d = filing.obj() # Returns TenD for CMBS, None for non-CMBS # Check if CMBS data is available if ten_d and ten_d.has_asset_data: loans = ten_d.loans # pandas DataFrame properties = ten_d.properties # pandas DataFrame summary = ten_d.asset_data.summary() # CMBSSummary` * * * Display ------- When you call `filing.obj()` on a CMBS Form 10-D filing, edgartools parses the filing and displays a rich panel with issuer information, distribution period, and asset data summary: ![TenD Display](https://edgartools.readthedocs.io/en/latest/images/tend-display.webp) The panel shows: - Issuing entity (ABS trust name and CIK) - ABS type (detected as CMBS, AUTO, CREDIT\_CARD, etc.) - Distribution period dates - Filing date - Depositor and sponsor entities - Security classes registered - Asset data summary (loans, properties, total balance, average rates/DSCR/occupancy, property types) * * * Core Data Structure ------------------- ### TenD (Top-Level Object) | Property | Type | Description | | --- | --- | --- | | `filing` | `Filing` | The underlying Filing object | | `form` | `str` | Form type (10-D or 10-D/A for amendments) | | `company` | `str` | Company name from the filing | | `filing_date` | `date` | Filing date | | `accession_number` | `str` | SEC accession number | | `issuing_entity` | `ABSEntity` | The ABS trust/entity (parsed from HTML header) | | `depositor` | `ABSEntity` | The depositor entity | | `sponsors` | `List[ABSEntity]` | Sponsor entities | | `distribution_period` | `DistributionPeriod` | Period covered (start\_date, end\_date) | | `security_classes` | `List[str]` | Classes of securities registered | | `abs_type` | `ABSType` | Detected ABS type enum (CMBS, AUTO, CREDIT\_CARD, RMBS, STUDENT\_LOAN, UTILITY, OTHER) | | `has_asset_data` | `bool` | Whether EX-102 XML exists (CMBS indicator) | | `asset_data` | `Optional[CMBSAssetData]` | Parsed CMBS asset data (lazy-loaded) | | `loans` | `Optional[DataFrame]` | Convenience property for `asset_data.loans` | | `properties` | `Optional[DataFrame]` | Convenience property for `asset_data.properties` | * * * Supporting Data Models ---------------------- ### ABSEntity Represents an entity involved in the ABS transaction (issuer, depositor, or sponsor). | Field | Type | Description | | --- | --- | --- | | `name` | `str` | Entity name | | `cik` | `Optional[str]` | SEC Central Index Key | | `file_number` | `Optional[str]` | Commission file number | ### DistributionPeriod The distribution period covered by the filing. | Field | Type | Description | | --- | --- | --- | | `start_date` | `Optional[date]` | Period start date | | `end_date` | `Optional[date]` | Period end date | ### ABSType (Enum) Detected asset-backed security type. EdgarTools detects this by looking for EX-102 XML (CMBS) or by analyzing company/issuer name keywords. | Value | Description | | --- | --- | | `CMBS` | Commercial Mortgage-Backed Securities | | `AUTO` | Auto Loan/Lease ABS | | `CREDIT_CARD` | Credit Card Receivables | | `RMBS` | Residential Mortgage-Backed Securities | | `STUDENT_LOAN` | Student Loan ABS | | `UTILITY` | Utility Securitizations | | `OTHER` | Other asset types | * * * CMBS Asset Data --------------- ### CMBSAssetData The `asset_data` property provides access to the structured CMBS loan and property data parsed from the EX-102 XML attachment. | Property | Type | Description | | --- | --- | --- | | `loans` | `DataFrame` | Loan-level data (49 fields) | | `properties` | DataFrame\` | Property-level data (46 fields) | | `summary()` | `CMBSSummary` | Aggregate statistics for the pool | ### Loans DataFrame (49 Fields) The `loans` DataFrame contains one row per loan with the following key columns: | Column | Type | Description | | --- | --- | --- | | `loan_id` | `str` | Prospectus loan identifier | | `originator` | `str` | Loan originator name | | `origination_date` | `date` | Date loan was originated | | `original_amount` | `float` | Original loan amount | | `original_term_months` | `int` | Original loan term in months | | `maturity_date` | `date` | Loan maturity date | | `original_rate` | `float` | Original interest rate | | `current_rate` | `float` | Current interest rate | | `actual_balance` | `float` | Current actual balance | | `scheduled_balance` | `float` | Scheduled principal balance at securitization | | `payment_status` | `str` | Payment status code (0=current, 1=30 days, etc.) | | `is_modified` | `bool` | Whether loan has been modified | | `is_balloon` | `bool` | Whether loan has balloon payment | | `is_interest_only` | `bool` | Whether loan is interest-only | | `primary_servicer` | `str` | Primary servicer name | | `lien_position` | `str` | Lien position code | | `num_properties` | `int` | Number of properties securing the loan | **Additional loan fields** include: `loan_id_type`, `period_start`, `period_end`, `securitization_rate`, `accrual_method`, `rate_type`, `io_term_months`, `first_payment_date`, `loan_structure`, `payment_type`, `payment_frequency`, `num_properties_securitization`, `grace_days`, `has_prepayment_premium`, `has_negative_amortization`, `lockout_end_date`, `yield_maintenance_end_date`, `prepayment_premium_end_date`, `period_begin_balance`, `scheduled_pi_due`, `servicer_fee_rate`, `scheduled_interest`, `scheduled_principal`, `unscheduled_principal`, `scheduled_end_balance`, `paid_through_date`, `servicing_advance_method`, `pi_advances_outstanding`, `ti_advances_outstanding`, `other_advances_outstanding`, `subject_to_demand`. ### Properties DataFrame (46 Fields) The `properties` DataFrame contains one or more rows per loan (one per property securing the loan) with the following key columns: | Column | Type | Description | | --- | --- | --- | | `loan_id` | `str` | Associated loan identifier | | `name` | `str` | Property name | | `address` | `str` | Street address | | `city` | `str` | City | | `state` | `str` | State code | | `zip` | `str` | ZIP code | | `county` | `str` | County | | `property_type` | `str` | Property type code (MF, OF, RT, etc.) | | `units` | `int` | Number of units/beds/rooms | | `sqft` | `int` | Net rentable square feet | | `year_built` | `int` | Year property was built | | `year_renovated` | `int` | Year last renovated | | `valuation` | `float` | Property valuation amount | | `valuation_source` | `str` | Valuation source code | | `valuation_date` | `date` | Valuation date | | `occupancy_securitization` | `float` | Occupancy % at securitization | | `occupancy_current` | `float` | Most recent occupancy % | | `revenue_securitization` | `float` | Revenue at securitization | | `opex_securitization` | `float` | Operating expenses at securitization | | `noi_securitization` | `float` | Net Operating Income at securitization | | `ncf_securitization` | `float` | Net Cash Flow at securitization | | `dscr_noi_securitization` | `float` | Debt Service Coverage Ratio (NOI) at securitization | | `dscr_ncf_securitization` | `float` | Debt Service Coverage Ratio (NCF) at securitization | **Additional property fields** include: `units_securitization`, `sqft_securitization`, `status`, `defeased_status`, `tenant_1_name`, `tenant_1_sqft`, `tenant_1_lease_exp`, `tenant_2_name`, `tenant_2_sqft`, `tenant_2_lease_exp`, `tenant_3_name`, `tenant_3_sqft`, `tenant_3_lease_exp`, `financials_date_securitization`, `financials_start_date`, `financials_end_date`, `revenue_current`, `opex_current`, `noi_current`, `ncf_current`, `debt_service_current`, `dscr_noi_current`, `dscr_ncf_current`. ### CMBSSummary The `summary()` method aggregates the loan and property data into pool-level statistics. | Field | Type | Description | | --- | --- | --- | | `num_loans` | `int` | Total number of loans | | `num_properties` | `int` | Total number of properties | | `total_loan_balance` | `float` | Sum of actual balances | | `total_original_loan_amount` | `float` | Sum of original amounts | | `avg_interest_rate` | `Optional[float]` | Average current interest rate | | `avg_dscr` | `Optional[float]` | Average DSCR (NOI-based) | | `avg_occupancy` | `Optional[float]` | Average occupancy % | | `property_types` | `Dict[str, int]` | Property type code → count | | `states` | `Dict[str, int]` | State → count | | `delinquent_loans` | `int` | Number of loans with payment\_status != '0' | | `modified_loans` | `int` | Number of modified loans | * * * Property Type Codes ------------------- Common values for `property_type` in the properties DataFrame: | Code | Description | | --- | --- | | `MF` | Multifamily | | `OF` | Office | | `RT` | Retail | | `IN` | Industrial | | `LO` | Lodging/Hotel | | `HC` | Healthcare | | `SS` | Self Storage | | `MH` | Manufactured Housing | | `OT` | Other | * * * Example: Analyzing a CMBS Pool ------------------------------ `from edgar import find # Find a CMBS 10-D filing (BANK5 2024-5YR9) filing = find('0001888524-25-020550') ten_d = filing.obj() # Display the filing print(ten_d) # Shows the rich panel with issuer, distribution period, and asset data summary # Check if CMBS data is available print(ten_d.has_asset_data) # True print(ten_d.abs_type) # ABSType.CMBS # Access loan data loans = ten_d.loans print(f"Number of loans: {len(loans)}") print(loans[['loan_id', 'actual_balance', 'current_rate', 'payment_status']].head()) # Access property data properties = ten_d.properties print(f"Number of properties: {len(properties)}") print(properties[['name', 'city', 'state', 'property_type', 'valuation']].head()) # Get pool summary summary = ten_d.asset_data.summary() print(f"Total balance: ${summary.total_loan_balance:,.0f}") print(f"Average rate: {summary.avg_interest_rate:.2%}") print(f"Average DSCR: {summary.avg_dscr:.2f}") print(f"Delinquent loans: {summary.delinquent_loans}") # Property type distribution for prop_type, count in sorted(summary.property_types.items(), key=lambda x: -x[1]): print(f" {prop_type}: {count}") # Top states by property count for state, count in sorted(summary.states.items(), key=lambda x: -x[1])[:5]: print(f" {state}: {count}")` * * * Example: Finding High-DSCR Properties ------------------------------------- `from edgar import find filing = find('0001888524-25-020550') ten_d = filing.obj() # Filter properties with strong debt service coverage properties = ten_d.properties high_dscr = properties[properties['dscr_noi_securitization'] > 1.5] print(f"Found {len(high_dscr)} properties with DSCR > 1.5") print(high_dscr[['name', 'property_type', 'state', 'dscr_noi_securitization']].sort_values('dscr_noi_securitization', ascending=False))` * * * Example: Tracking Delinquencies ------------------------------- `from edgar import find filing = find('0001888524-25-020550') ten_d = filing.obj() # Find delinquent loans (payment_status != '0') loans = ten_d.loans delinquent = loans[loans['payment_status'] != '0'] print(f"Delinquent loans: {len(delinquent)} out of {len(loans)}") if len(delinquent) > 0: print(delinquent[['loan_id', 'actual_balance', 'payment_status', 'primary_servicer']])` * * * Example: Geographic Distribution -------------------------------- `from edgar import find import pandas as pd filing = find('0001888524-25-020550') ten_d = filing.obj() # Aggregate property value by state properties = ten_d.properties state_totals = properties.groupby('state').agg({ 'valuation': 'sum', 'loan_id': 'count' }).rename(columns={'loan_id': 'num_properties'}) state_totals = state_totals.sort_values('valuation', ascending=False) print(state_totals.head(10))` * * * Searching for 10-D Filings -------------------------- `from edgar import get_filings, Company # Search for all recent 10-D filings recent_10d = get_filings(form="10-D").head(20) for filing in recent_10d: ten_d = filing.obj() # Only CMBS filings will have asset_data if ten_d and ten_d.has_asset_data: print(f"{ten_d.issuing_entity.name}: {ten_d.abs_type.value}") summary = ten_d.asset_data.summary() print(f" {summary.num_loans} loans, ${summary.total_loan_balance:,.0f}") # Search for 10-D filings by a specific company company = Company("1888524") # BANK as Depositor filings = company.get_filings(form="10-D")` * * * Important Notes --------------- 1. **CMBS-only support**: `filing.obj()` returns `TenD` only for CMBS filings with EX-102 XML asset data. Non-CMBS 10-D filings return `None`. 2. **DataFrames are pandas DataFrames**: You can use standard pandas operations to filter, aggregate, and analyze the data. 3. **Lazy loading**: The XML asset data is parsed on first access to `asset_data`, `loans`, or `properties`. 4. **Missing data**: Many fields can be `None` or `NaN`. Always handle missing data gracefully when doing calculations. 5. **Distribution report HTML parsing is not supported**: The narrative distribution report section has highly variable HTML formats across issuers and was found to have only ~42% extraction accuracy. EdgarTools focuses on the structured EX-102 XML data. * * * Data Source ----------- The structured CMBS data comes from the **EX-102 XML attachment** filed with the 10-D. The XML follows the SEC EDGAR schema: `http://www.sec.gov/edgar/document/absee/cmbs/assetdata` The issuer, depositor, sponsor, distribution period, and security class information is parsed from the HTML header of the 10-D filing. * * * Common Use Cases ---------------- ### Portfolio Risk Analysis * Identify loans with low DSCR or high vacancy * Track delinquency rates across servicers * Monitor properties in specific states or metros ### Investment Research * Compare pool composition across issuers * Analyze property type concentrations * Evaluate geographic diversification ### Compliance Monitoring * Track modified loans * Monitor payment status trends * Review servicer performance ### Market Intelligence * Analyze securitization activity by sponsor * Track originator market share * Compare interest rates across pools Back to top --- # Sale Notices (Form 144) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/form144-data-object-guide/#form-144-parse-insider-restricted-stock-sale-notices) Form 144: Parse Insider Restricted Stock Sale Notices ===================================================== Form 144 is a notice of proposed sale of restricted securities under SEC Rule 144. Filed by company insiders (officers, directors, 10%+ shareholders) before selling restricted or control securities. This guide details all data available from the `Form144` class for building views. * * * Overview -------- | Property | Type | Description | | --- | --- | --- | | Class Name | `Form144` | | | Forms Handled | `144`, `144/A` | | | Module | `edgar.form144` | | | Source Data | XML document | | * * * Basic Metadata -------------- | Property | Type | Description | Example | | --- | --- | --- | --- | | `filing_date` | `str` | Date filed with SEC | `"2023-04-18"` | | `is_amendment` | `bool` | Whether this is a 144/A amendment | `False` | * * * Seller Information ------------------ | Property | Type | Description | Example | | --- | --- | --- | --- | | `person_selling` | `str` | Name of person selling | `"John Smith"` | | `relationships` | `List[str]` | Relationship(s) to issuer | `["Director", "Officer"]` | | `address` | `Address` | Seller's address | See Address object | | `filer` | `Filer` | SEC filer credentials | CIK, entity name, file number | | `contact` | `Contact` | Contact information | Name, phone, email | ### Address Object | Property | Type | Description | | --- | --- | --- | | `street1` | `str` | Street line 1 | | `street2` | `str` | Street line 2 | | `city` | `str` | City | | `state_or_country` | `str` | State/country code | | `zipcode` | `str` | ZIP code | * * * Issuer Information ------------------ | Property | Type | Description | Example | | --- | --- | --- | --- | | `issuer_name` | `str` | Company name | `"Owens Corning"` | | `issuer_cik` | `str` | Company CIK | `"1370946"` | | `sec_file_number` | `str` | SEC file number | `"001-36390"` | | `issuer_contact_phone` | `str` | Issuer contact phone | `"419-248-8000"` | | `company` | `Company` | Full Company object | Lazy-loaded from CIK | * * * Securities Information (Proposed Sale) -------------------------------------- The core data about the securities to be sold. ### Quick Access Properties (Single Security) For filings with one security type (most common): | Property | Type | Description | Example | | --- | --- | --- | --- | | `units_to_be_sold` | `int` | Total shares to sell | `3000` | | `market_value` | `float` | Total market value | `300000.0` | | `security_class` | `str` | Security type | `"Common"` | | `approx_sale_date` | `str` | Approximate sale date | `"04/18/2023"` | | `exchange_name` | `str` | Exchange for sale | `"NYSE"` | | `broker_name` | `str` | Broker executing sale | `"Fidelity Brokerage Services LLC"` | ### Aggregation Properties (Multi-Security) For filings with multiple security types: | Property | Type | Description | | --- | --- | --- | | `total_units_to_be_sold` | `int` | Sum across all securities | | `total_market_value` | `float` | Sum across all securities | | `num_securities` | `int` | Count of security types | | `is_multi_security` | `bool` | Whether multiple security types | ### Securities Information Holder Access the full DataFrame via `securities_info`: `form144 = filing.obj() # Holder object with aggregation methods holder = form144.securities_info # Properties holder.empty # bool - is data empty? holder.total_units_to_be_sold # int - sum of all units holder.total_market_value # float - sum of market values holder.security_classes # List[str] - all security types holder.exchanges # List[str] - unique exchanges holder.brokers # List[str] - unique brokers holder.percent_of_outstanding # float - % of outstanding shares holder.avg_price_per_unit # float - market_value / units # Iteration for row in holder: print(row.security_class, row.units_to_be_sold) # Raw DataFrame df = form144.securities_information # pd.DataFrame` ### Securities Information DataFrame Columns | Column | Type | Description | | --- | --- | --- | | `security_class` | `str` | Security type (Common, Preferred, etc.) | | `units_to_be_sold` | `int` | Number of units | | `market_value` | `float` | Aggregate market value | | `units_outstanding` | `int` | Total outstanding shares | | `approx_sale_date` | `str` | Expected sale date (MM/DD/YYYY) | | `exchange_name` | `str` | Exchange code (NYSE, NASDAQ, etc.) | | `broker_name` | `str` | Broker/market maker name | * * * Securities To Be Sold (Acquisition History) ------------------------------------------- Details about how/when the seller acquired the securities. ### Access `# Holder with aggregation holder = form144.securities_selling holder.empty # bool holder.total_amount_acquired # int - sum of all acquired holder.acquisition_dates # List[str] - unique dates holder.has_gift_transactions # bool - any gifts? # Iteration for row in holder: print(row.acquired_date, row.amount_acquired) # Raw DataFrame df = form144.securities_to_be_sold` ### Securities To Be Sold DataFrame Columns | Column | Type | Description | | --- | --- | --- | | `security_class` | `str` | Security type | | `acquired_date` | `str` | Date acquired (MM/DD/YYYY) | | `amount_acquired` | `int` | Shares acquired | | `nature_of_acquisition` | `str` | How acquired (e.g., "Employee Stock Award") | | `acquired_from` | `str` | Person/entity acquired from | | `is_gift` | `str` | Gift transaction? ("Y"/"N") | | `donar_acquired_date` | `str` | If gift, when donor acquired | | `payment_date` | `str` | Date payment made | | `nature_of_payment` | `str` | Payment type (e.g., "CASH") | * * * Securities Sold Past 3 Months ----------------------------- Prior sales within the last 90 days (SEC Rule 144 volume limits). ### Access `# Holder with aggregation holder = form144.recent_sales holder.empty # bool holder.total_amount_sold # int - sum of amounts holder.total_gross_proceeds # float - sum of proceeds holder.sellers # List[str] - unique seller names # Iteration for row in holder: print(row.sale_date, row.amount_sold, row.gross_proceeds) # Raw DataFrame df = form144.securities_sold_past_3_months # Flag for no activity form144.nothing_to_report # bool - no sales in past 3 months` ### Aggregation Properties | Property | Type | Description | | --- | --- | --- | | `total_amount_sold_past_3_months` | `int` | Total shares sold recently | | `total_gross_proceeds_past_3_months` | `float` | Total proceeds | ### Securities Sold Past 3 Months DataFrame Columns | Column | Type | Description | | --- | --- | --- | | `security_class` | `str` | Security type | | `seller_name` | `str` | Seller name | | `sale_date` | `str` | Date of sale (MM/DD/YYYY) | | `amount_sold` | `int` | Shares sold | | `gross_proceeds` | `float` | Sale proceeds | * * * Analyst Metrics (Computed Properties) ------------------------------------- Pre-computed metrics for investment analysis: ### Percentage Metrics | Property | Type | Description | | --- | --- | --- | | `percent_of_holdings` | `float` | % of outstanding shares being sold | | `avg_price_per_unit` | `float` | Market value / units to sell | ### Holding Period Analysis | Property | Type | Description | | --- | --- | --- | | `holding_period_days` | `int` or `None` | Avg days held before sale | | `holding_period_years` | `float` or `None` | Avg years held | ### 10b5-1 Plan Compliance | Property | Type | Description | | --- | --- | --- | | `is_10b5_1_plan` | `bool` | Sale under 10b5-1 trading plan | | `days_since_plan_adoption` | `int` or `None` | Days from plan adoption to sale | | `cooling_off_compliant` | `bool` or `None` | 90-day cooling off observed (post-2022 rule) | ### Anomaly Detection Flags | Property | Type | Description | | --- | --- | --- | | `is_large_liquidation` | `bool` | Selling >5% of outstanding | | `is_short_hold` | `bool` | Holding period <1 year | | `has_multiple_plans` | `bool` | Multiple 10b5-1 plan dates | | `anomaly_flags` | `List[str]` | All triggered flags | **Possible Anomaly Flags:** - `LARGE_LIQUIDATION` - Selling >5% of outstanding shares - `SHORT_HOLD` - Held <1 year before selling - `COOLING_OFF_VIOLATION` - <90 days since 10b5-1 plan adoption - `MULTIPLE_PLANS` - Multiple 10b5-1 plan adoption dates * * * Notice Signature ---------------- | Property | Type | Description | | --- | --- | --- | | `notice_signature.notice_date` | `str` | Date notice signed | | `notice_signature.signature` | `str` | Signature text | | `notice_signature.plan_adoption_dates` | `List[str]` | 10b5-1 plan dates | * * * Remarks ------- | Property | Type | Description | | --- | --- | --- | | `remarks` | `str` | Additional remarks from filer | * * * Summary Methods --------------- ### get\_summary() Returns dict with key filing info: `summary = form144.get_summary() # Returns: { 'person_selling': str, 'issuer': str, 'issuer_cik': str, 'relationships': List[str], 'num_securities': int, 'total_units_to_be_sold': int, 'total_market_value': float, 'security_classes': List[str], 'exchanges': List[str], 'nothing_to_report_past_3_months': bool, 'total_sold_past_3_months': int, 'is_amendment': bool, 'filing_date': str, }` ### to\_analyst\_summary() Returns dict optimized for investment screening: `summary = form144.to_analyst_summary() # Returns: { # Identity 'person_selling': str, 'issuer': str, 'issuer_cik': str, 'relationships': List[str], 'filing_date': str, # Sale metrics 'units_to_sell': int, 'market_value': float, 'percent_of_holdings': float, 'avg_price_per_unit': float, # Timing 'sale_date': str, 'holding_period_years': float or None, # 10b5-1 Plan 'is_10b5_1_plan': bool, 'days_since_plan_adoption': int or None, 'cooling_off_compliant': bool or None, # Recent activity 'sold_past_3_months': int, 'proceeds_past_3_months': float, # Flags 'anomaly_flags': List[str], # Metadata 'is_amendment': bool, 'exchange': str, 'broker': str, }` ### to\_dataframe() Returns DataFrame with one row per security: `df = form144.to_dataframe() # Columns: security_class, units_to_be_sold, market_value, units_outstanding, # approx_sale_date, exchange_name, broker_name, person_selling, # issuer, issuer_cik, filing_date, is_amendment` * * * View Design Recommendations --------------------------- ### Primary View Components 1. **Header Section** 2. Issuer name + CIK 3. Form type (144 or 144/A amendment) 4. Filing date 5. Amendment indicator if applicable 6. **Seller Panel** 7. Person selling (prominent) 8. Relationship(s) to issuer (badges/tags) 9. Contact info (collapsible) 10. **Sale Summary Card** 11. Units to be sold (large, highlighted) 12. Market value 13. % of holdings 14. Avg price per unit 15. Approximate sale date 16. Exchange / Broker 17. **Compliance Panel** 18. 10b5-1 Plan: Yes/No indicator 19. Days since plan adoption 20. Cooling off status (green checkmark or red warning) 21. Anomaly flags (warning badges) 22. **Securities Information Table** 23. Security class, units, market value, sale date 24. Exchange, broker for each 25. **Acquisition History Table** (collapsible) 26. How/when securities were acquired 27. Gift transaction indicators 28. **Recent Sales Table** (collapsible) 29. Past 3 months activity 30. Or "Nothing to report" message 31. **Signature Footer** 32. Notice date 33. Signature 34. Plan adoption dates ### Data Priority for Display | Priority | Data | Reason | | --- | --- | --- | | High | Person selling + relationships | Who is selling | | High | Units, market value, % holdings | Sale magnitude | | High | Anomaly flags | Risk indicators | | Medium | 10b5-1 compliance | Regulatory context | | Medium | Sale date, exchange, broker | Transaction details | | Medium | Recent sales (past 3 months) | Volume limit context | | Low | Acquisition history | Background detail | | Low | Full address, contact | Administrative | ### Visual Indicators (Suggested) | Condition | Visual Treatment | | --- | --- | | `is_amendment` | Yellow "Amendment" badge | | `is_large_liquidation` | Red warning icon | | `is_short_hold` | Orange warning icon | | `cooling_off_compliant == False` | Red "Violation" badge | | `is_10b5_1_plan` | Blue "10b5-1" badge | | Large `market_value` (>$1M) | Emphasized styling | ### Anomaly Flags Color Coding | Flag | Color | Meaning | | --- | --- | --- | | `LARGE_LIQUIDATION` | Red | \>5% of company being sold | | `SHORT_HOLD` | Orange | Held less than 1 year | | `COOLING_OFF_VIOLATION` | Red | 10b5-1 rule violation | | `MULTIPLE_PLANS` | Yellow | Unusual plan activity | * * * Example Data Structure ---------------------- `{ # Identity "person_selling": "Brian O. Chambers", "issuer_name": "Owens Corning", "issuer_cik": "1370946", "relationships": ["Officer"], "filing_date": "2023-04-18", "is_amendment": False, # Sale summary "units_to_be_sold": 3000, "market_value": 300000.0, "percent_of_holdings": 0.0028, "avg_price_per_unit": 100.0, "security_class": "Common", "approx_sale_date": "04/18/2023", "exchange_name": "NYSE", "broker_name": "Fidelity Brokerage Services LLC", # Compliance "is_10b5_1_plan": True, "days_since_plan_adoption": 120, "cooling_off_compliant": True, "holding_period_years": 2.5, # Flags "anomaly_flags": [], # Recent activity "nothing_to_report": True, "total_sold_past_3_months": 0, "total_proceeds_past_3_months": 0.0, # Securities info (array) "securities_information": [ { "security_class": "Common", "units_to_be_sold": 3000, "market_value": 300000.0, "units_outstanding": 107000000, "approx_sale_date": "04/18/2023", "exchange_name": "NYSE", "broker_name": "Fidelity Brokerage Services LLC" } ], # Acquisition history (array) "securities_to_be_sold": [ { "security_class": "Common", "acquired_date": "03/15/2021", "amount_acquired": 500, "nature_of_acquisition": "Employee Stock Award", "acquired_from": "Issuer", "is_gift": "N" } // ... more entries ], # Signature "notice_date": "04/17/2023", "signature": "/s/ Brian O. Chambers", "plan_adoption_dates": ["01/15/2023"] }` * * * Notes for Implementation ------------------------ 1. **XML-Only**: Form 144 data comes exclusively from XML. Filings before ~2015 may not have XML and will return `None` from `filing.obj()`. 2. **Multiple Securities**: Some filings contain multiple security types. Always use aggregation properties (`total_*`) or iterate through `securities_info`. 3. **Placeholder Dates**: SEC forms use `01/01/1933` as placeholder dates. The class filters these out in computed metrics. 4. **Holding Period**: Calculated from acquisition dates to sale date. Returns `None` if dates are invalid or missing. 5. **Cooling Off Rule**: The 90-day requirement took effect in 2022. Earlier filings may show "violations" that weren't violations at the time. Back to top --- # Form Types - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/FormType-Quick-Reference/#formtype-quick-reference-guide) FormType Quick Reference Guide ============================== 🚀 **Getting Started** ---------------------- `from edgar import Company from edgar.enums import FormType company = Company("AAPL") # New: IDE autocomplete for form types filings = company.get_filings(form=FormType.ANNUAL_REPORT) # Old: Still works perfectly filings = company.get_filings(form="10-K")` 📋 **All Available FormTypes** ------------------------------ ### **Periodic Reports** `FormType.ANNUAL_REPORT # "10-K" FormType.QUARTERLY_REPORT # "10-Q" FormType.ANNUAL_REPORT_AMENDED # "10-K/A" FormType.QUARTERLY_REPORT_AMENDED # "10-Q/A" FormType.FOREIGN_ANNUAL # "20-F" FormType.CANADIAN_ANNUAL # "40-F" FormType.EMPLOYEE_BENEFIT_PLAN # "11-K"` ### **Current Reports** `FormType.CURRENT_REPORT # "8-K" FormType.FOREIGN_CURRENT_REPORT # "6-K"` ### **Proxy Statements** `FormType.PROXY_STATEMENT # "DEF 14A" FormType.PRELIMINARY_PROXY # "PRE 14A" FormType.ADDITIONAL_PROXY # "DEFA14A" FormType.MERGER_PROXY # "DEFM14A"` ### **Registration Statements** `FormType.REGISTRATION_S1 # "S-1" FormType.REGISTRATION_S3 # "S-3" FormType.REGISTRATION_S4 # "S-4" FormType.REGISTRATION_S8 # "S-8" FormType.FOREIGN_REGISTRATION_F1 # "F-1" FormType.FOREIGN_REGISTRATION_F3 # "F-3" FormType.FOREIGN_REGISTRATION_F4 # "F-4"` ### **Prospectuses** `FormType.PROSPECTUS_424B1 # "424B1" FormType.PROSPECTUS_424B2 # "424B2" FormType.PROSPECTUS_424B3 # "424B3" FormType.PROSPECTUS_424B4 # "424B4" FormType.PROSPECTUS_424B5 # "424B5"` ### **Ownership Reports** `FormType.BENEFICIAL_OWNERSHIP_13D # "SC 13D" FormType.BENEFICIAL_OWNERSHIP_13G # "SC 13G"` ### **Other Important Forms** `FormType.SPECIALIZED_DISCLOSURE # "SD" FormType.ASSET_BACKED_SECURITIES # "ARS" FormType.LATE_10K_NOTICE # "NT 10-K" FormType.LATE_10Q_NOTICE # "NT 10-Q"` 📚 **Form Collections** ----------------------- `from edgar.enums import PERIODIC_FORMS, PROXY_FORMS, REGISTRATION_FORMS # Pre-defined collections for common workflows PERIODIC_FORMS # [10-K, 10-Q, 10-K/A, 10-Q/A] PROXY_FORMS # [DEF 14A, PRE 14A, DEFA14A, DEFM14A] REGISTRATION_FORMS # [S-1, S-3, S-4, S-8]` ⚡ **Usage Examples** -------------------- ### **Basic Usage** `# Annual reports with autocomplete annual_filings = company.get_filings(form=FormType.ANNUAL_REPORT) # Quarterly reports quarterly_filings = company.get_filings(form=FormType.QUARTERLY_REPORT) # Current reports (8-Ks) current_filings = company.get_filings(form=FormType.CURRENT_REPORT)` ### **Combined Filters** `# Recent annual reports filings = company.get_filings( form=FormType.ANNUAL_REPORT, year=[2022, 2023] ) # Proxy statements this year proxies = company.get_filings( form=FormType.PROXY_STATEMENT, year=2023 )` ### **Multiple Form Types** `# Mix FormType and strings filings = company.get_filings(form=[ FormType.ANNUAL_REPORT, FormType.QUARTERLY_REPORT, "8-K" # String still works ]) # Using form collections periodic_filings = company.get_filings(form=PERIODIC_FORMS)` 🛡️ **Error Handling** ---------------------- `# Typos get helpful suggestions try: filings = company.get_filings(form="10k") # Missing hyphen except ValueError as e: print(e) # "Invalid form type '10k'. Use FormType enum for autocomplete..."` 🔄 **Migration Guide** ---------------------- ### **No Breaking Changes** `# ALL existing code works unchanged: company.get_filings(form="10-K") # ✅ Works company.get_filings(form=["10-K", "10-Q"]) # ✅ Works company.get_filings(form="8-K", year=2023) # ✅ Works` ### **Gradual Adoption** `# Option 1: Keep using strings filings = company.get_filings(form="10-K") # Option 2: Migrate to FormType for autocomplete filings = company.get_filings(form=FormType.ANNUAL_REPORT) # Option 3: Mix as convenient filings = company.get_filings(form=[FormType.ANNUAL_REPORT, "8-K"])` 💡 **IDE Benefits** ------------------- * **Autocomplete**: Type `FormType.` to see all 31 options * **Documentation**: Hover over enums to see SEC form codes * **Type Safety**: mypy/PyCharm catches invalid form parameters * **Refactoring**: Find all usages of specific form types 🔗 **Links** ------------ * **GitHub Discussion**: [#423 Type Hinting Implementation](https://github.com/dgunning/edgartools/discussions/423) * **Feature Branch**: `feat/strenum-type-hinting` * **Test Files**: Run `python formtype_demo_examples.py` for live examples * * * _Perfect backwards compatibility + modern Python typing = Happy developers! 🎉_ Back to top --- # Getting XBRL Data - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/getting-xbrl/#getting-xbrl-data-from-sec-filings) Getting XBRL Data from SEC Filings ================================== Overview -------- The `edgar.xbrl` module provides a powerful yet user-friendly API for processing **XBRL (eXtensible Business Reporting Language)** financial data from SEC filings. Key Features ------------ * **Intuitive API**: Access financial statements with simple, readable method calls * **Multi-period Analysis**: Compare financial data across quarters and years with statement stitching * **Standardized Concepts**: View company-specific terms or standardized labels for cross-company comparison * **Rich Rendering**: Display beautifully formatted financial statements in console or notebooks * **Smart Period Selection**: Automatically identify and select relevant periods for meaningful comparisons * **DataFrame Export**: Convert any statement to pandas DataFrames for further analysis Getting Started --------------- You can get the XBRL from a single filing, or stitch together multiple filings. ### Getting XBRL from a single filing For a single filing you can use `filing.xbrl()` to get the XBRL data, and then access the financial and other statements. `from edgar import Company from edgar.xbrl.xbrl import XBRL # Get a company's latest 10-K filing company = Company('AAPL') filing = company.latest("10-K") # Parse XBRL data xb = filing.xbrl() # Access statements through the user-friendly API statements = xb.statements # Display financial statements balance_sheet = statements.balance_sheet() income_statement = statements.income_statement() cash_flow = statements.cashflow_statement()` ### Getting XBRL from multiple filings You can also stitch together multiple filings to create a multi-period view of financial statements. This uses the `edgar.XBRLS` class to combine data across multiple filings. Each filing should be of the same type (e.g., all 10-Ks or all 10-Qs) and from the same company. `from edgar import Company from edgar.xbrl import XBRLS # Get multiple filings for trend analysis company = Company('AAPL') filings = company.get_filings(form="10-K").head(3) # Get the last 3 annual reports # Create a stitched view across multiple filings xbrls = XBRLS.from_filings(filings) # Access stitched statements stitched_statements = xbrls.statements # Display multi-period statements income_trend = stitched_statements.income_statement() balance_sheet_trend = stitched_statements.balance_sheet() cashflow_trend = stitched_statements.cashflow_statement() # Use view="detailed" to include dimensional breakdowns across periods income_detailed = stitched_statements.income_statement(view="detailed")` User-Friendly Features ---------------------- ### Simple Statement Access Access common financial statements with intuitive methods: `# Get basic statements balance_sheet = statements.balance_sheet() income_statement = statements.income_statement() cash_flow = statements.cashflow_statement() statement_of_equity = statements.statement_of_equity() # Access any statement by type comprehensive_income = statements["ComprehensiveIncome"]` ### Smart Period Views Choose from intelligent period selection views: `# See available period views period_views = statements.get_period_views("IncomeStatement") for view in period_views: print(f"- {view['name']}: {view['description']}") # Render with specific view annual_comparison = statements.income_statement(period_view="Annual Comparison") quarter_comparison = statements.income_statement(period_view="Quarterly Comparison")` ### Easy Conversion to DataFrames Transform any statement into a pandas DataFrame for further analysis: `# Get DataFrame of income statement df = income_statement.to_dataframe()` Statement Stitching for Trend Analysis -------------------------------------- The XBRLS class combines data from multiple periods with intelligent handling of concept changes: `# Create stitched statements across multiple filings xbrls = XBRLS.from_filings(filings) stitched = xbrls.statements # Get a three-year comparison of income statements income_trend = stitched.income_statement(max_periods=3) # Convert to DataFrame for time series analysis trend_df = income_trend.to_dataframe()` Rendering Options ----------------- The XBRL2 module provides flexible output options for financial statements: `# Display with default styling as Rich tables in console/notebooks print(statements.balance_sheet()) # Show full date ranges for duration periods print(statements.income_statement(show_date_range=True)) # Customize period view print(statements.income_statement(period_view="Annual Comparison")) # Convert to pandas DataFrame for analysis df = statements.to_dataframe("BalanceSheet") # Export the statement to markdown income_statement = statements.income_statement() markdown_text = income_statement.render().to_markdown()` ### Statement Display Options The rendering system offers several customization options: | Option | Description | | --- | --- | | `standard=True` | Add `standard_concept` metadata for cross-company analysis (default). Labels remain as company-reported. | | `standard=False` | Skip standardization metadata entirely | | `show_date_range=True` | Show complete date ranges for duration periods (e.g., "Jan 1 - Mar 31, 2023") | | `show_date_range=False` | Show only end dates for cleaner presentation (default) | | `period_view="Name"` | Select a predefined period view ("Annual Comparison", "Quarterly Comparison", etc.) | | `period_filter="duration_..."` | Filter to a specific period by period key | > **Note**: Labels always show the company's original presentation. The `standard_concept` column maps each line item to a standard category (e.g., "Revenue", "CommonEquity") for filtering and cross-company aggregation. Use `df.groupby('standard_concept').sum()` to aggregate by standard concepts. ### The `RenderedStatement` Class The `render_statement()` function returns a `RenderedStatement` object, which provides multiple output formats: `# Get a rendered statement statement = xbrl.render_statement("BalanceSheet") # Display as Rich table (default) print(statement) # Convert to pandas DataFrame df = statement.to_dataframe() # Export to markdown markdown = statement.to_markdown()` ### Customizing Statement Appearance The rendering engine automatically handles: * Proper monetary formatting with scale indicators (thousands, millions, billions) * Appropriate indentation for statement hierarchy * Formatting of section headers and dimension items * Correct display of share counts and per-share values * Fiscal period indicators in statement titles * Unit notes (e.g., "In millions, except per share data") For stitched multi-period statements, you can control periods, date formatting, and dimensional detail: `# Get 3-year comparison with full date ranges annual_trend = stitched_statements.income_statement( max_periods=3, show_date_range=True ) # Include dimensional breakdowns (e.g., cost by segment across years) detailed_trend = stitched_statements.income_statement(view="detailed")` Advanced Features ----------------- ### Custom Period Selection `# Get specific periods from available options available_periods = xbrl.reporting_periods latest_period = available_periods[0] # Render with specific period if latest_period['type'] == 'instant': period_filter = f"instant_{latest_period['date']}" latest_balance_sheet = statements.balance_sheet().render(period_filter=period_filter)` ### Statement Data Exploration `# Get raw statement data for custom processing raw_data = statements.balance_sheet().get_raw_data() # Extract specific information assets = [item for item in raw_data if 'assets' in item['label'].lower()]` Design Philosophy ----------------- The XBRL2 module is designed with these principles: 1. **User-First API**: Simple methods that match how financial analysts think about statements 2. **Intelligent Defaults**: Smart period selection and formatting that "just works" out of the box 3. **Flexible Output Options**: Rich tables for display, DataFrames for analysis, and raw data for custom processing 4. **Consistency Across Companies**: Standardized concepts that enable cross-company comparison Period Selection Logic ---------------------- The XBRL2 module implements sophisticated period selection logic to ensure appropriate periods are displayed for financial statements: ### Quarterly Statement Period Selection When rendering quarterly statements (when fiscal\_period\_focus is Q1, Q2, Q3, or Q4): 1. The system identifies true quarterly periods by filtering duration periods to those with 80-100 day durations 2. If quarterly periods are found, the most recent one is selected as the current quarter 3. For comparison, the system looks for periods with similar duration from approximately 1-2 years prior 4. If no quarterly periods are found, it falls back to the most recent period with a warning ### Annual Statement Period Selection For annual reports (when fiscal\_period\_focus is FY): 1. Annual periods are identified by looking for ~365 day durations or fiscal year markers 2. The system prioritizes periods that align with the entity's fiscal year end 3. Up to three most recent fiscal years are displayed in chronological order This intelligent period selection ensures appropriate periods are displayed for statements, with robust fallbacks when ideal periods aren't available. Enhanced Facts API ------------------ The XBRL2 module includes a powerful facts query interface for direct access to individual XBRL facts: `from edgar import Company from edgar.xbrl import XBRL # Parse XBRL data company = Company('AAPL') filing = company.get_filings(form='10-K').latest() xbrl = XBRL.from_filing(filing) # Access the facts view facts = xbrl.facts # Query facts by various attributes revenue = facts.query().by_concept('Revenue').to_dataframe() balance_sheet_facts = facts.query().by_statement_type('BalanceSheet').to_dataframe() # Use predefined period views - returns important metadata including available periods income_views = facts.get_available_period_views('IncomeStatement') for view in income_views: print(f"- {view['name']}: {view['description']} ({view['facts_count']} facts)") # Get facts filtered by period view annual_comparison = facts.get_facts_by_period_view('IncomeStatement', 'Annual Comparison') # Flexible text search across all text fields (concept, label, element name) earnings_facts = facts.search_facts("Earnings Per Share") # Filter by period keys - useful for custom period selection facts.query().by_period_keys(['duration_2023-01-01_2023-12-31', 'duration_2022-01-01_2022-12-31']).to_dataframe() # Query dimensional data facts_by_segment = facts.query().by_dimension('Segment').to_dataframe() # Safe numeric value filtering with proper None handling large_income_items = (facts.query() .by_statement_type('IncomeStatement') .by_value(lambda v: v > 1_000_000_000) .sort_by('numeric_value', ascending=False) .to_dataframe()) # Time series analysis revenue_over_time = facts.time_series('Revenue')` XBRL Calculation Support ------------------------ The XBRL2 module properly handles calculation relationships from XBRL calculation linkbases: `# Values are automatically adjusted according to calculation weights # For example, elements with negative weights (-1.0) like "IncreaseDecreaseInInventories" # are automatically negated to maintain proper calculation relationships cash_flow_statement = statements.cashflow_statement() # The calculation trees are accessible for inspection for role_uri, calc_tree in xbrl.calculation_trees.items(): print(f"Calculation tree: {calc_tree.definition}") for element_id, node in calc_tree.all_nodes.items(): if node.weight != 1.0: print(f"- {element_id}: weight={node.weight}")` The parser intelligently applies calculation weights to ensure consistent financial data presentation: 1. **Expense Concept Consistency**: Major expense categories (R&D, SG&A, Marketing, etc.) are consistently positive across companies, matching SEC CompanyFacts API behavior 2. **Cash Flow Integrity**: Elements with negative weights (-1.0) in cash flow statements maintain proper sign relationships for accurate calculations 3. **Legitimate Negatives Preserved**: Concepts that should be negative (tax benefits, foreign exchange gains/losses) retain their intended signs 4. **Cross-Company Comparability**: Eliminates inconsistencies where MSFT showed R&D as negative while AAPL showed positive values `# Example: R&D expenses are now consistently positive across companies msft_statements = msft_xbrl.statements.income_statement() aapl_statements = aapl_xbrl.statements.income_statement() # Both show R&D as positive values for proper comparison msft_rnd = msft_statements.get_concept_value("ResearchAndDevelopmentExpense") # $32.5B (positive) aapl_rnd = aapl_statements.get_concept_value("ResearchAndDevelopmentExpense") # $31.4B (positive)` Need help building an XBRL pipeline? The code above extracts XBRL data for one company. Scaling to thousands — with taxonomy normalization, custom extension mapping, and multi-year consistency — is where it gets hard. * **[XBRL consulting for AI & data teams →](https://www.edgar.tools/consulting/xbrl?utm_source=edgartools-docs&utm_medium=see-live&utm_content=xbrl-consulting) ** * **[See all SEC data consulting services →](https://www.edgar.tools/consulting?utm_source=edgartools-docs&utm_medium=see-live&utm_content=xbrl-consulting) ** From the creator of edgartools. [Book a call →](https://www.edgar.tools/consulting/xbrl?utm_source=edgartools-docs&utm_medium=see-live&utm_content=xbrl-consulting#contact) Future Enhancements ------------------- * Enhanced support for non-standard financial statements * Interactive visualization options * Expanded dimensional analysis capabilities * Automatic footnote association * Financial ratio calculations * Advanced calculation validation and reconciliation Back to top --- # Municipal Advisors (MA-I) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/mai-data-object-guide/#municipal-advisor-form-ma-i-data-object-guide) Municipal Advisor Form (MA-I) Data Object Guide =============================================== Overview -------- **Form MA-I** is an SEC filing required for individuals who work as municipal advisors. Municipal advisors provide advice to state and local governments on bond issuances and other municipal financial products. The SEC requires registration of these individuals to protect municipalities from unqualified or unethical advisors. The `MunicipalAdvisorForm` class in edgartools parses MA-I XML filings into structured Python objects, making it easy to extract applicant information, employment history, and critically, disclosure information about any regulatory or legal issues. Access Pattern -------------- `from edgar import Filing # Get an MA-I filing filing = Filing(form="MA-I", ...) # Parse into MunicipalAdvisorForm object ma_form = filing.obj()` * * * Core Data Structure ------------------- ### MunicipalAdvisorForm (Top-Level Object) | Property | Type | Description | | --- | --- | --- | | `filing` | `Filing` | Reference to the original SEC filing | | `filer` | `Filer` | CIK and CCC of the filing entity | | `is_amendment` | `bool` | Whether this is an amendment (MA-I/A) | | `is_individual` | `bool` | Whether applicant is an individual | | `previous_accession_no` | `str` | Accession number of prior filing if amendment | | `contact` | `Contact` | Filing contact information | | `applicant` | `Applicant` | The individual applying for registration | | `internet_notification_addresses` | `List[str]` | Email addresses for notifications | | `municipal_advisor_offices` | `List[MunicipalAdvisorOffice]` | Firms where individual is employed | | `employment_history` | `EmploymentHistory` | Current and previous employment | | `disclosures` | `Disclosures` | All disclosure questions and answers | | `signature` | `Signature` | Filing signature | * * * Applicant Information --------------------- ### Applicant The individual seeking municipal advisor registration. | Property | Type | Description | UI Usage | | --- | --- | --- | --- | | `name` | `Name` | Full legal name | Primary display | | `other_names` | `List[Name]` | Aliases, maiden names, etc. | Name history | | `crd` | `str` | FINRA CRD number | BrokerCheck link | | `number_of_advisory_firms` | `int` | Count of associated MA firms | Employment scope | | `full_name` | `str` (property) | Concatenated full name | Display convenience | ### Name | Property | Type | Description | | --- | --- | --- | | `first_name` | `str` | First name | | `middle_name` | `str` | Middle name | | `last_name` | `str` | Last name | | `suffix` | `str` | Name suffix (Jr., III, etc.) | | `full_name` | `str` (property) | Complete formatted name | ### Contact Filing contact person (may differ from applicant). | Property | Type | Description | | --- | --- | --- | | `name` | `str` | Contact person name | | `phone` | `str` | Phone number | | `email` | `str` | Email address | ### Filer | Property | Type | Description | | --- | --- | --- | | `cik` | `str` | SEC Central Index Key | | `ccc` | `str` | EDGAR Filer ID confirmation code | * * * Municipal Advisor Offices ------------------------- ### MunicipalAdvisorOffice Firms where the individual works as a municipal advisor. | Property | Type | Description | UI Usage | | --- | --- | --- | --- | | `cik` | `str` | Firm's SEC CIK | Link to firm filings | | `firm_name` | `str` | Legal name of MA firm | Firm display | | `is_independent_relationship` | `bool` | Independent contractor flag | Employment type badge | | `recent_employment_commenced_date` | `str` | When employment started | Employment timeline | | `file_number` | `str` | SEC registration file number | Registration link | | `offices` | `List[Office]` | Physical office locations | Location display | ### Office Physical office locations where the individual works. | Property | Type | Description | | --- | --- | --- | | `start_date` | `str` | When work at this location began | | `location_info` | `str` | Additional location details | | `address` | `Address` | Full address | | `street1` | `str` (property) | Street address line 1 | | `street2` | `str` (property) | Street address line 2 | | `city` | `str` (property) | City | | `state_or_country` | `str` (property) | State/country code | | `zipcode` | `str` (property) | Postal code | * * * Employment History ------------------ ### EmploymentHistory Complete work history relevant to municipal advisory activities. | Property | Type | Description | | --- | --- | --- | | `current_employer` | `Employer` | Current employment | | `previous_employers` | `List[Employer]` | Past 10 years of employment | ### Employer | Property | Type | Description | UI Usage | | --- | --- | --- | --- | | `name` | `str` | Employer name | Company display | | `start_date` | `str` | Employment start (formatted as "Jun 2015") | Timeline | | `end_date` | `str` | Employment end (None if current) | Timeline | | `ma_related` | `bool` | Municipal advisor related work | MA badge | | `investment_related` | `bool` | Investment related work | Investment badge | | `position` | `str` | Job title/position | Role display | | `address` | `Address` | Employer location | Location context | * * * Disclosures (Critical Section) ------------------------------ The disclosures section is the most important part of MA-I filings for due diligence. It contains yes/no answers to extensive questions about the applicant's regulatory, legal, and financial history. ### Disclosures (Container) | Property | Type | Description | | --- | --- | --- | | `criminal_disclosure` | `CriminalDisclosure` | Criminal history questions | | `regulatory_disclosure` | `RegulatoryDisclosure` | SEC/CFTC regulatory history | | `civil_disclosure` | `CivilDisclosure` | Civil court proceedings | | `complaint_disclosure` | `ComplaintDisclosure` | Customer complaints | | `termination_disclosure` | `TerminationDisclosure` | Employment terminations | | `financial_disclosure` | `FinancialDisclosure` | Bankruptcy/financial issues | | `judgement_lien_disclosure` | `JudgementLienDisclosure` | Judgments and liens | | `investigation_disclosure` | `InvestigationDisclosure` | Ongoing investigations | | `any()` | method | Returns `True` if ANY disclosure is positive | * * * ### CriminalDisclosure Criminal history questions. | Property | Type | Description | Red Flag Level | | --- | --- | --- | --- | | `is_convicted_of_felony` | `bool` | Felony conviction | Critical | | `is_charged_with_felony` | `bool` | Pending felony charges | Critical | | `is_org_convicted_of_felony` | `bool` | Caused org felony conviction | High | | `is_org_charged_with_felony` | `bool` | Caused org felony charges | High | | `is_convicted_of_misdemeanor` | `bool` | Misdemeanor conviction (investment-related) | Medium | | `is_charged_with_misdemeanor` | `bool` | Pending misdemeanor charges | Medium | | `is_org_convicted_of_misdemeanor` | `bool` | Caused org misdemeanor conviction | Medium | | `is_org_charged_with_misdemeanor` | `bool` | Caused org misdemeanor charges | Medium | | `any()` | method | Returns `True` if any criminal disclosure | \- | * * * ### RegulatoryDisclosure SEC, CFTC, and other regulatory agency actions. | Property | Type | Description | Red Flag Level | | --- | --- | --- | --- | | `is_made_false_statement` | `bool` | Made false statement to regulator | Critical | | `is_violated_regulation` | `bool` | Violated SEC/CFTC regulation | High | | `is_cause_of_denial` | `bool` | Caused denial of registration | High | | `is_order_against` | `bool` | Order entered against individual | High | | `is_imposed_penalty` | `bool` | Penalty imposed | High | | `is_un_ethical` | `bool` | Found dishonest or unethical | Critical | | `is_found_in_violation_of_regulation` | `bool` | Found in violation | High | | `is_found_in_cause_of_denial` | `bool` | Found to cause denial | High | | `is_order_against_activity` | `bool` | Order against activities | High | | `is_denied_license` | `bool` | License denied/suspended/revoked | Critical | | `is_found_made_false_statement` | `bool` | Found to have made false statement | Critical | | `is_found_in_violation_of_rules` | `bool` | Found in violation of rules | High | | `is_found_in_cause_of_suspension` | `bool` | Caused suspension | High | | `is_discipliend` | `bool` | Disciplined (expelled/barred) | Critical | | `is_authorized_to_act_attorney` | `bool` | Attorney authorization suspended | Medium | | `is_regulatory_complaint` | `bool` | Regulatory complaint pending | Medium | | `is_violated_security_act` | `bool` | Violated Securities Act | Critical | | `is_will_fully_aided` | `bool` | Willfully aided violation | Critical | | `is_failed_to_supervise` | `bool` | Failed to supervise | High | | `is_found_will_fully_aided` | `bool` | Found to willfully aid violation | Critical | | `is_association_bared` | `bool` | Barred from association | Critical | | `is_final_order` | `bool` | Final order entered against | High | | `is_will_fully_violated_security_act` | `bool` | Willfully violated Securities Act | Critical | | `is_failed_resonably` | `bool` | Failed reasonably to supervise | High | | `any()` | method | Returns `True` if any regulatory disclosure | \- | * * * ### CivilDisclosure Civil court proceedings. | Property | Type | Description | Red Flag Level | | --- | --- | --- | --- | | `is_enjoined` | `bool` | Enjoined in connection with MA business | High | | `is_found_violation_of_regulation` | `bool` | Court found violation | High | | `is_dismissed` | `bool` | Civil action dismissed with settlement | Medium | | `is_named_in_civil_proceeding` | `bool` | Currently named in civil proceeding | Medium | | `any()` | method | Returns `True` if any civil disclosure | \- | * * * ### ComplaintDisclosure Customer and regulatory complaints. | Property | Type | Description | Red Flag Level | | --- | --- | --- | --- | | `is_complaint_pending` | `bool` | MA-related complaint pending | Medium | | `is_complaint_settled` | `bool` | MA-related complaint settled | Low | | `is_fraud_case_pending` | `bool` | Fraud case pending | High | | `is_fraud_case_resulting_award` | `bool` | Fraud case resulted in award | High | | `is_fraud_case_settled` | `bool` | Fraud case settled | Medium | | `any()` | method | Returns `True` if any complaint disclosure | \- | * * * ### TerminationDisclosure Employment terminations under adverse circumstances. | Property | Type | Description | Red Flag Level | | --- | --- | --- | --- | | `is_violated_industry_standards` | `bool` | Terminated for violating standards | High | | `is_involved_in_fraud` | `bool` | Terminated for fraud involvement | Critical | | `is_failed_to_supervise` | `bool` | Terminated for supervision failure | High | | `any()` | method | Returns `True` if any termination disclosure | \- | * * * ### FinancialDisclosure Financial problems within past 10 years. | Property | Type | Description | Red Flag Level | | --- | --- | --- | --- | | `is_compromised` | `bool` | Made compromise with creditors | Medium | | `is_bankruptcy_petition` | `bool` | Organization filed bankruptcy | Medium | | `is_trustee_appointed` | `bool` | Trustee appointed for organization | Medium | | `is_bond_revoked` | `bool` | Bonding company denied/revoked bond | High | | `any()` | method | Returns `True` if any financial disclosure | \- | * * * ### JudgementLienDisclosure | Property | Type | Description | Red Flag Level | | --- | --- | --- | --- | | `is_lien_against` | `bool` | Currently has judgment liens | Medium | | `any()` | method | Returns `True` if any lien disclosure | \- | * * * ### InvestigationDisclosure | Property | Type | Description | Red Flag Level | | --- | --- | --- | --- | | `is_investigated` | `bool` | Currently under investigation | High | * * * Signature --------- ### Signature | Property | Type | Description | | --- | --- | --- | | `signature` | `str` | Signature text | | `date_signed` | `str` | Date of signature | | `title` | `str` | Signer's title | * * * UI Component Recommendations ---------------------------- ### Applicant Summary Card * Full name prominently displayed * CRD number (link to FINRA BrokerCheck) * Number of advisory firms * Amendment status badge * Other names (expandable) ### Disclosure Summary Panel (Critical) **This is the most important UI element for due diligence.** Display a traffic-light style indicator: - **Green**: No disclosures (`disclosures.any() == False`) - **Red**: Has disclosures (`disclosures.any() == True`) If disclosures exist, show breakdown by category: - Criminal (red if any) - Regulatory (red if any) - Civil (yellow if any) - Complaints (yellow if any) - Terminations (orange if any) - Financial (yellow if any) - Liens (yellow if any) - Under Investigation (red if true) ### Employment History Timeline * Visual timeline showing employment progression * Badges for "MA Related" and "Investment Related" work * Current employer highlighted * Employer names and positions ### Municipal Advisor Firm Details * Firm name and CIK * SEC file number (link to SEC) * Independent contractor badge * Office locations (expandable) ### Disclosure Detail Panels For each disclosure category, show individual questions with Yes/No answers: - Group by severity (Critical, High, Medium, Low) - Highlight any "Yes" answers prominently - Provide context/explanation for each question * * * Common Queries and Filters -------------------------- For SAAS features, consider enabling filters by: 1. **Clean Record** - Filter where `disclosures.any() == False` 2. **Has Disclosures** - Filter where `disclosures.any() == True` 3. **Criminal Issues** - Filter by `criminal_disclosure.any()` 4. **Regulatory Issues** - Filter by `regulatory_disclosure.any()` 5. **Under Investigation** - Filter by `investigation_disclosure.is_investigated` 6. **MA Firm** - Filter by `municipal_advisor_offices[].firm_name` 7. **State** - Filter by office state 8. **Amendment vs New** - Filter by `is_amendment` * * * Due Diligence Use Cases ----------------------- ### Red Flag Detection `# Check for any disclosures if ma_form.disclosures.any(): print("WARNING: Applicant has disclosures") # Check specific high-severity items if ma_form.disclosures.criminal_disclosure.is_convicted_of_felony: print("CRITICAL: Felony conviction") if ma_form.disclosures.regulatory_disclosure.is_association_bared: print("CRITICAL: Barred from association")` ### Employment Verification `# Get current employer current = ma_form.employment_history.current_employer print(f"Currently at: {current.name} as {current.position}") # Check MA experience for employer in ma_form.employment_history.previous_employers: if employer.ma_related: print(f"MA experience at: {employer.name}")` ### Firm Association `# List all MA firms for office in ma_form.municipal_advisor_offices: print(f"Firm: {office.firm_name}") print(f" CIK: {office.cik}") print(f" File #: {office.file_number}") print(f" Independent: {office.is_independent_relationship}")` * * * Data Quality Notes ------------------ 1. **Disclosure answers** - All stored as booleans; `True` indicates a positive disclosure 2. **Employment dates** - Formatted as "Mon YYYY" (e.g., "Jun 2015") 3. **CRD numbers** - May be empty for new registrants 4. **Other names** - May include maiden names, aliases, prior legal names 5. **Multiple offices** - An individual may work at multiple MA firm offices * * * Example Data Access ------------------- `# Get filing and parse ma_form = filing.obj() # Access applicant info print(ma_form.applicant.full_name) print(f"CRD: {ma_form.applicant.crd}") # Check for disclosures (most important!) if ma_form.disclosures.any(): print("Has disclosures - review required") # Check specific categories if ma_form.disclosures.criminal_disclosure.any(): print(" - Criminal disclosures present") if ma_form.disclosures.regulatory_disclosure.any(): print(" - Regulatory disclosures present") else: print("Clean record - no disclosures") # Employment history print(f"Current employer: {ma_form.employment_history.current_employer.name}") for emp in ma_form.employment_history.previous_employers: print(f" Previous: {emp.name} ({emp.start_date} - {emp.end_date})") # MA firm associations for office in ma_form.municipal_advisor_offices: print(f"Associated with: {office.firm_name}") # Signature verification print(f"Signed by: {ma_form.signature.signature} on {ma_form.signature.date_signed}")` * * * Form Variants ------------- The `MunicipalAdvisorForm` class handles these form types: | Form | Description | | --- | --- | | `MA-I` | Individual municipal advisor initial registration | | `MA-I/A` | Amendment to individual registration | | `MA` | Firm municipal advisor registration | | `MA/A` | Amendment to firm registration | Check `filing.form` or `is_amendment` to determine the filing type. Back to top --- # Crowdfunding (Form C) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/formc-data-object-guide/#form-c-parse-crowdfunding-offerings-regulation-cf) Form C: Parse Crowdfunding Offerings (Regulation CF) ==================================================== Overview -------- **Form C** is an SEC filing used by companies raising capital through crowdfunding under Regulation Crowdfunding (Regulation CF). These filings provide insights into small business fundraising campaigns, including offering terms, issuer financials, and funding portal information. The `FormC` class in edgartools parses Form C XML filings into structured Python objects, making it easy to track crowdfunding campaigns, analyze issuer financials, and monitor offering progress. What Form C Covers ------------------ Form C encompasses several filing types that track the complete lifecycle of a crowdfunding offering: * **Form C** - Initial crowdfunding offering * **Form C/A** - Amendment to the offering * **Form C-U** - Progress update during the offering period * **Form C-AR** - Annual report (filed yearly after offering closes) * **Form C-TR** - Termination report (when offering is withdrawn) Access Pattern -------------- `from edgar import get_filings # Get recent Form C filings filing = get_filings(form="C").head(1)[0] # Parse into FormC object formc = filing.obj() # Access offering details print(f"{formc.issuer_name}") print(f"Target: ${formc.offering_information.target_amount:,.0f}") print(f"Maximum: ${formc.offering_information.maximum_offering_amount:,.0f}") print(f"Deadline: {formc.offering_information.deadline_date}")` When you call `filing.obj()` on a Form C filing, edgartools displays a rich formatted panel showing all filing details: ![Form C Display](https://edgartools.readthedocs.io/en/latest/images/formc-display.webp) * * * Core Data Structure ------------------- ### FormC (Top-Level Object) | Property | Type | Description | | --- | --- | --- | | `filer_information` | `FilerInformation` | CIK, filing period, test/live flag | | `issuer_information` | `IssuerInformation` | Company name, address, website, legal status, jurisdiction | | `offering_information` | `Optional[OfferingInformation]` | Offering terms (present in C, C-U; absent in C-AR, C-TR) | | `annual_report_disclosure` | `Optional[AnnualReportDisclosure]` | Financial disclosures (present in C-AR) | | `signature_info` | `SignatureInfo` | Officer and director signatures | | `form` | `str` | Form type (C, C/A, C-U, C-AR, C-TR) | | `description` | `str` | Human-readable form description | | `issuer_name` | `str` | Convenience property for issuer name | | `portal_name` | `Optional[str]` | Funding portal name (if present) | | `campaign_status` | `str` | Derived status (Active, Terminated, Annual Report, etc.) | | `days_to_deadline` | `Optional[int]` | Days until offering deadline (negative if expired) | | `is_expired` | `bool` | Whether offering deadline has passed | ### Computed Properties | Property | Returns | Description | | --- | --- | --- | | `issuer` | `IssuerCompany` | Company wrapper with offering-specific methods | | `docs` | `Docs` | Access comprehensive API documentation | ### Methods | Method | Returns | Description | | --- | --- | --- | | `get_offering()` | `Offering` | Complete offering lifecycle (all related filings) | | `to_context(detail)` | `str` | Token-efficient text representation for AI context | * * * Offering Information -------------------- The `offering_information` property contains the core fundraising terms. It is present in Form C and Form C-U but **not** in Form C-AR or C-TR. ### OfferingInformation | Property | Type | Description | | --- | --- | --- | | `compensation_amount` | `str` | Portal fees and compensation structure | | `security_offered_type` | `Optional[str]` | Type of security (Equity, Debt, Other) | | `security_offered_other_desc` | `Optional[str]` | Description if type is "Other" | | `offering_amount` | `Optional[float]` | Target offering amount in dollars | | `maximum_offering_amount` | `Optional[float]` | Maximum offering amount (for over-subscription) | | `price` | `Optional[str]` | Price per security (as string) | | `no_of_security_offered` | `Optional[str]` | Number of securities offered (as string) | | `deadline_date` | `Optional[date]` | Offering deadline | | `over_subscription_accepted` | `Optional[str]` | "Y" or "N" | | `over_subscription_allocation_type` | `Optional[str]` | How over-subscriptions are allocated | | `price_determination_method` | `Optional[str]` | How price was determined | ### Computed Properties | Property | Returns | Description | | --- | --- | --- | | `security_description` | `str` | Combined type and description | | `target_amount` | `Optional[float]` | Alias for `offering_amount` | | `offering_deadline` | `Optional[date]` | Alias for `deadline_date` | | `price_per_security` | `Optional[float]` | Price parsed as float | | `number_of_securities` | `Optional[int]` | Number of securities as int | | `percent_to_maximum` | `Optional[float]` | Target as percentage of maximum | ### Example: Analyzing Offering Terms `from edgar import get_filings filing = get_filings(form="C").head(1)[0] formc = filing.obj() offering = formc.offering_information if offering: print(f"Security: {offering.security_description}") print(f"Target: ${offering.target_amount:,.0f}") print(f"Maximum: ${offering.maximum_offering_amount:,.0f}") print(f"Target is {offering.percent_to_maximum:.0f}% of maximum") if offering.price_per_security and offering.number_of_securities: print(f"Price: ${offering.price_per_security:.2f} per unit") print(f"Units: {offering.number_of_securities:,}") # Check deadline days_left = formc.days_to_deadline if days_left is not None: if days_left > 0: print(f"Deadline: {days_left} days remaining") else: print(f"Offering expired {abs(days_left)} days ago")` * * * Financial Disclosures --------------------- The `annual_report_disclosure` property contains financial statements for the current and prior fiscal year. It is present in **Form C-AR** (annual reports) and sometimes in Form C. ### AnnualReportDisclosure | Property | Type | Description | | --- | --- | --- | | `current_employees` | `int` | Number of employees | | `total_asset_most_recent_fiscal_year` | `float` | Total assets (current year) | | `total_asset_prior_fiscal_year` | `float` | Total assets (prior year) | | `cash_equi_most_recent_fiscal_year` | `float` | Cash and equivalents (current) | | `cash_equi_prior_fiscal_year` | `float` | Cash and equivalents (prior) | | `act_received_most_recent_fiscal_year` | `float` | Accounts receivable (current) | | `act_received_prior_fiscal_year` | `float` | Accounts receivable (prior) | | `short_term_debt_most_recent_fiscal_year` | `float` | Short-term debt (current) | | `short_term_debt_prior_fiscal_year` | `float` | Short-term debt (prior) | | `long_term_debt_most_recent_fiscal_year` | `float` | Long-term debt (current) | | `long_term_debt_prior_fiscal_year` | `float` | Long-term debt (prior) | | `revenue_most_recent_fiscal_year` | `float` | Revenue (current) | | `revenue_prior_fiscal_year` | `float` | Revenue (prior) | | `cost_goods_sold_most_recent_fiscal_year` | `float` | Cost of goods sold (current) | | `cost_goods_sold_prior_fiscal_year` | `float` | Cost of goods sold (prior) | | `tax_paid_most_recent_fiscal_year` | `float` | Taxes paid (current) | | `tax_paid_prior_fiscal_year` | `float` | Taxes paid (prior) | | `net_income_most_recent_fiscal_year` | `float` | Net income (current) | | `net_income_prior_fiscal_year` | `float` | Net income (prior) | | `offering_jurisdictions` | `List[str]` | States where offering is available | ### Computed Financial Metrics | Property | Returns | Description | | --- | --- | --- | | `total_debt_most_recent` | `float` | Short-term + long-term debt (current year) | | `total_debt_prior` | `float` | Short-term + long-term debt (prior year) | | `debt_to_asset_ratio` | `Optional[float]` | Debt-to-asset ratio as percentage | | `revenue_growth_yoy` | `Optional[float]` | Year-over-year revenue growth percentage | | `asset_growth_yoy` | `Optional[float]` | Year-over-year asset growth percentage | | `is_pre_revenue` | `bool` | True if no revenue in current year | | `burn_rate_change` | `Optional[float]` | Change in net income (shows burn rate trend) | ### Convenience Aliases (Most Recent Year) For easier access to current year data: | Property | Returns | Maps To | | --- | --- | --- | | `total_assets` | `float` | `total_asset_most_recent_fiscal_year` | | `cash_and_cash_equivalents` | `float` | `cash_equi_most_recent_fiscal_year` | | `accounts_receivable` | `float` | `act_received_most_recent_fiscal_year` | | `short_term_debt` | `float` | `short_term_debt_most_recent_fiscal_year` | | `long_term_debt` | `float` | `long_term_debt_most_recent_fiscal_year` | | `revenues` | `float` | `revenue_most_recent_fiscal_year` | | `cost_of_goods_sold` | `float` | `cost_goods_sold_most_recent_fiscal_year` | | `taxes_paid` | `float` | `tax_paid_most_recent_fiscal_year` | | `net_income` | `float` | `net_income_most_recent_fiscal_year` | | `number_of_employees` | `int` | `current_employees` | ### Example: Analyzing Issuer Financials `from edgar import get_filings # Get a Form C-AR (annual report) filing = get_filings(form="C-AR").head(1)[0] formc_ar = filing.obj() financials = formc_ar.annual_report_disclosure if financials: # Basic metrics print(f"Employees: {financials.number_of_employees}") print(f"Total Assets: ${financials.total_assets:,.0f}") print(f"Cash: ${financials.cash_and_cash_equivalents:,.0f}") # Revenue analysis if financials.is_pre_revenue: print("Status: Pre-revenue") else: print(f"Revenue: ${financials.revenues:,.0f}") if financials.revenue_growth_yoy is not None: print(f"Revenue Growth: {financials.revenue_growth_yoy:+.1f}% YoY") # Profitability print(f"Net Income: ${financials.net_income:,.0f}") # Debt analysis total_debt = financials.total_debt_most_recent if total_debt > 0: print(f"Total Debt: ${total_debt:,.0f}") if financials.debt_to_asset_ratio: print(f"Debt-to-Asset Ratio: {financials.debt_to_asset_ratio:.0f}%")` * * * Issuer Information ------------------ ### IssuerInformation The `issuer_information` property contains company details. | Property | Type | Description | | --- | --- | --- | | `name` | `str` | Legal name of the issuing company | | `address` | `Address` | Business address | | `website` | `str` | Company website | | `co_issuer` | `bool` | Whether there is a co-issuer | | `funding_portal` | `Optional[FundingPortal]` | Crowdfunding portal intermediary | | `legal_status` | `str` | Legal structure (Corporation, LLC, etc.) | | `jurisdiction` | `str` | State/country of incorporation | | `date_of_incorporation` | `date` | Date company was formed | ### FundingPortal The intermediary platform facilitating the crowdfunding raise. | Property | Type | Description | | --- | --- | --- | | `name` | `str` | Portal name (e.g., "StartEngine", "Wefunder") | | `cik` | `str` | SEC Central Index Key for the portal | | `crd` | `Optional[str]` | FINRA CRD number | | `file_number` | `str` | SEC commission file number for the portal | ### IssuerCompany The `issuer` property returns an `IssuerCompany` object that provides offering-specific methods: | Method | Returns | Description | | --- | --- | --- | | `as_company()` | `Company` | Convert to full Company object | | `get_offerings()` | `List[Offering]` | All crowdfunding offerings by this issuer | | `latest_offering()` | `Optional[Offering]` | Most recent offering | ### Example: Working with Issuers `from edgar import get_filings filing = get_filings(form="C").head(1)[0] formc = filing.obj() # Basic issuer info print(f"Company: {formc.issuer_name}") print(f"Incorporated: {formc.issuer_information.jurisdiction} " f"({formc.issuer_information.date_of_incorporation.year})") print(f"Website: {formc.issuer_information.website}") # Funding portal if formc.portal_name: print(f"Portal: {formc.portal_name}") # Get all offerings by this company issuer = formc.issuer offerings = issuer.get_offerings() print(f"{issuer.name} has {len(offerings)} crowdfunding offerings") # Convert to full Company object for more data company = issuer.as_company() print(f"All SEC filings: {len(company.get_filings())}")` * * * Offering Lifecycle ------------------ Each crowdfunding campaign goes through multiple filing stages. The `get_offering()` method returns an `Offering` object that tracks the complete lifecycle. ### Example: Tracking an Offering Over Time `from edgar import get_filings # Get a Form C filing filing = get_filings(form="C").head(1)[0] formc = filing.obj() # Get the complete offering lifecycle offering = formc.get_offering() # View all filings for this offering print(offering.timeline()) # Access specific filing types initial_filing = offering.initial() # Original Form C updates = offering.updates() # Form C-U progress updates annual_reports = offering.annual_reports() # Form C-AR reports amendments = offering.amendments() # Form C/A amendments` * * * Form Variants and Data Availability ----------------------------------- Different Form C variants contain different data sections: | Form | `offering_information` | `annual_report_disclosure` | `funding_portal` | | --- | --- | --- | --- | | **C** | Yes | Sometimes | Yes | | **C/A** | Yes | Sometimes | Yes | | **C-U** | Limited | Sometimes | Yes | | **C-AR** | No | Yes | No | | **C-TR** | No | No | Usually minimal | **Key patterns:** * **Form C** (initial offering): Has full offering terms and portal info * **Form C-U** (progress update): May have limited offering info updates * **Form C-AR** (annual report): Has financial disclosures, no offering terms * **Form C-TR** (termination): Minimal data, indicates offering withdrawn Always check for `None` before accessing `offering_information` or `annual_report_disclosure`: `if formc.offering_information: print(f"Target: ${formc.offering_information.target_amount:,.0f}") if formc.annual_report_disclosure: print(f"Revenue: ${formc.annual_report_disclosure.revenues:,.0f}")` * * * Signature Information --------------------- ### SignatureInfo | Property | Type | Description | | --- | --- | --- | | `issuer_signature` | `IssuerSignature` | Company signature | | `signatures` | `List[PersonSignature]` | Individual officer/director signatures | | `signers` | `List[Signer]` | Consolidated list of unique signers | ### IssuerSignature | Property | Type | Description | | --- | --- | --- | | `issuer` | `str` | Company name as signed | | `title` | `str` | Title of signer (e.g., "Chief Executive Officer") | | `signature` | `str` | Signature as signed | ### PersonSignature | Property | Type | Description | | --- | --- | --- | | `signature` | `str` | Person's name as signed | | `title` | `str` | Title (e.g., "Director") | | `date` | `date` | Date signed | * * * Common Use Cases ---------------- ### Finding Active Offerings `from edgar import get_filings # Get recent Form C filings active_offerings = get_filings(form="C").head(20) for filing in active_offerings: formc = filing.obj() # Filter for non-expired offerings if not formc.is_expired and formc.offering_information: offering = formc.offering_information print(f"{formc.issuer_name}") print(f" Target: ${offering.target_amount:,.0f}") print(f" Deadline: {offering.deadline_date}") print(f" Days left: {formc.days_to_deadline}") print()` ### Analyzing Issuer Financial Health `from edgar import get_filings # Get Form C-AR annual reports annual_reports = get_filings(form="C-AR").head(10) for filing in annual_reports: formc = filing.obj() fin = formc.annual_report_disclosure if fin: print(f"{formc.issuer_name}") print(f" Assets: ${fin.total_assets:,.0f}") print(f" Cash: ${fin.cash_and_cash_equivalents:,.0f}") if fin.is_pre_revenue: print(f" Status: Pre-revenue") else: print(f" Revenue: ${fin.revenues:,.0f}") if fin.revenue_growth_yoy: print(f" Growth: {fin.revenue_growth_yoy:+.1f}% YoY") print(f" Net Income: ${fin.net_income:,.0f}") print()` ### Tracking a Company's Crowdfunding History `from edgar import Company # Get a company that has done crowdfunding company = Company("1881570") # Example: ViiT Health # Get all crowdfunding filings cf_filings = company.get_filings(form=['C', 'C/A', 'C-U', 'C-AR', 'C-TR']) # Group by offering from edgar.offerings.formc import group_offerings_by_file_number grouped = group_offerings_by_file_number(cf_filings) print(f"{company.name} has {len(grouped)} crowdfunding offerings") for file_num, offering_filings in grouped.items(): print(f"\nOffering {file_num}:") for filing in offering_filings: print(f" {filing.form:8} - {filing.filing_date}")` * * * AI-Optimized Context -------------------- The `to_context()` method provides a token-efficient text representation optimized for AI/LLM context windows: `from edgar import get_filings filing = get_filings(form="C").head(1)[0] formc = filing.obj() # Minimal detail (~100-200 tokens) print(formc.to_context(detail='minimal')) # Standard detail (~300-500 tokens) print(formc.to_context(detail='standard')) # Full detail (~600-1000 tokens) print(formc.to_context(detail='full'))` This is useful when passing Form C data to language models or building AI-powered analysis tools. * * * Quick Reference --------------- ### Properties Quick Lookup | Access | Returns | Example | | --- | --- | --- | | `formc.issuer_name` | `str` | Company name | | `formc.portal_name` | `Optional[str]` | Funding portal | | `formc.campaign_status` | `str` | "Active", "Terminated", etc. | | `formc.days_to_deadline` | `Optional[int]` | Days remaining (negative if expired) | | `formc.is_expired` | `bool` | Deadline passed? | | `formc.offering_information` | `Optional[OfferingInformation]` | Offering terms | | `formc.annual_report_disclosure` | `Optional[AnnualReportDisclosure]` | Financial data | | `formc.issuer` | `IssuerCompany` | Company wrapper with offering methods | ### Methods Quick Lookup | Method | Returns | Use Case | | --- | --- | --- | | `formc.get_offering()` | `Offering` | Get complete offering lifecycle | | `formc.to_context(detail)` | `str` | Token-efficient text for AI | | `formc.issuer.get_offerings()` | `List[Offering]` | All offerings by this company | | `formc.issuer.as_company()` | `Company` | Convert to full Company object | * * * Things to Know -------------- **Monetary amounts are floats.** All dollar amounts are parsed as floats and can be used directly in calculations. **Check for None before accessing optional sections.** The `offering_information` and `annual_report_disclosure` fields may be `None` depending on the form type. **Deadlines can be in the past.** Use `days_to_deadline` to check if an offering is still active. Negative values mean the deadline has passed. **Pre-revenue companies are common.** Many crowdfunding issuers have zero revenue. Use `is_pre_revenue` to detect this. **Portal file numbers identify the portal, not the offering.** To track a specific offering, use the issuer file number from `filing.file_number` or the `Offering` object. **Form C-AR has no offering information.** Annual reports focus on financial disclosures and don't include the original offering terms. **State jurisdictions are abbreviated.** The `jurisdiction` field uses state codes like "DE", "CA", "NY". * * * Related ------- * **User Guide:** Working with Form C Filings (coming soon) * **API Reference:** `edgar.offerings.formc` module * **Related Filings:** [Form D: Private Placements](https://edgartools.readthedocs.io/en/latest/guides/formd-data-object-guide/) * **Company Data:** [Find a Company](https://edgartools.readthedocs.io/en/latest/guides/finding-companies/) Back to top --- # Proxy Statements (DEF 14A) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/proxystatement-data-object-guide/#proxy-statements-def-14a-parse-executive-compensation-and-governance-data) Proxy Statements (DEF 14A): Parse Executive Compensation and Governance Data ============================================================================ Form DEF 14A is a definitive proxy statement filed by public companies before annual shareholder meetings. It contains critical information about executive compensation, board composition, shareholder voting matters, and corporate governance. This guide details all data available from the `ProxyStatement` class for building views. * * * Overview -------- | Property | Type | Description | | --- | --- | --- | | Class Name | `ProxyStatement` | | | Forms Handled | `DEF 14A`, `DEFA14A`, `DEFM14A`, `DEF 14A/A` | | | Module | `edgar.proxy` | | | Source Data | XBRL (primary) + HTML (secondary) | | ### Form Type Descriptions | Form | Description | | --- | --- | | `DEF 14A` | Definitive Proxy Statement - standard proxy filing | | `DEFA14A` | Definitive Additional Proxy Soliciting Materials | | `DEFM14A` | Definitive Proxy Statement relating to Merger or Acquisition | | `DEF 14A/A` | Amendment to Definitive Proxy Statement | ### Data Source Reliability | Source | Reliability | Description | | --- | --- | --- | | XBRL | High | Executive compensation, pay vs performance - standardized across all companies | | HTML | Medium | Beneficial ownership, board info, proposals - requires parsing | * * * Basic Metadata -------------- | Property | Type | Description | Example | | --- | --- | --- | --- | | `form` | `str` | Form type | `"DEF 14A"` | | `filing_date` | `str` | Date filed with SEC | `"2025-01-10"` | | `fiscal_year_end` | `str` | Fiscal year end date | `"2024-09-28"` | | `company_name` | `str` | Company legal name | `"Apple Inc."` | | `cik` | `str` | Central Index Key | `"0000320193"` | | `accession_number` | `str` | SEC accession number | `"0001308179-25-000008"` | * * * Executive Compensation (XBRL - High Reliability) ------------------------------------------------ Executive compensation data is extracted from XBRL using the SEC's Executive Compensation Disclosure (ECD) taxonomy. This data is highly standardized and available for all companies. ### PEO (Principal Executive Officer / CEO) | Property | Type | XBRL Concept | Description | Example | | --- | --- | --- | --- | --- | | `peo_name` | `str` | `ecd:PeoName` | CEO name | `"Mr. Cook"` | | `peo_total_comp` | `Decimal` | `ecd:PeoTotalCompAmt` | Total compensation from Summary Compensation Table | `74,609,802` | | `peo_actually_paid_comp` | `Decimal` | `ecd:PeoActuallyPaidCompAmt` | Compensation Actually Paid (CAP) | `168,980,568` | ### Non-PEO Named Executive Officers (NEOs) | Property | Type | XBRL Concept | Description | Example | | --- | --- | --- | --- | --- | | `neo_avg_total_comp` | `Decimal` | `ecd:NonPeoNeoAvgTotalCompAmt` | Average NEO total compensation | `27,178,896` | | `neo_avg_actually_paid_comp` | `Decimal` | `ecd:NonPeoNeoAvgCompActuallyPaidAmt` | Average NEO compensation actually paid | `58,633,525` | ### Compensation Time Series (5 Years) The `executive_compensation` property returns a DataFrame with 5 years of compensation data: `proxy = filing.obj() comp_df = proxy.executive_compensation # pd.DataFrame` | Column | Type | Description | | --- | --- | --- | | `fiscal_year_end` | `date` | End of fiscal year | | `peo_total_comp` | `Decimal` | PEO total from SCT | | `peo_actually_paid_comp` | `Decimal` | PEO compensation actually paid | | `neo_avg_total_comp` | `Decimal` | Non-PEO NEO average total | | `neo_avg_actually_paid_comp` | `Decimal` | Non-PEO NEO average CAP | ### Example Output `# Apple Inc. Executive Compensation (5 years) fiscal_year_end peo_total_comp peo_actually_paid_comp neo_avg_total_comp neo_avg_actually_paid 2024-09-28 74,609,802 168,980,568 27,178,896 58,633,525 2023-09-30 63,209,845 106,643,588 26,938,240 48,892,163 2022-09-24 99,420,097 128,833,021 26,929,095 35,842,114 2021-09-25 98,734,394 311,845,801 26,989,456 89,764,231 2020-09-26 14,769,259 4,567,123 23,976,158 12,589,743` ### Named Executives (Dimensional Data) Some companies tag individual executive data using dimensional XBRL. When available: `# Check if individual executive data is available if proxy.has_individual_executive_data: executives = proxy.named_executives # list of executive dicts for exec in executives: print(f"{exec['name']}: ${exec['actually_paid_comp']:,}")` | Property | Type | Description | | --- | --- | --- | | `has_individual_executive_data` | `bool` | Whether individual executive dimensions are available | | `named_executives` | `list[dict]` | Individual executive compensation details (when available) | **Note**: Only ~60% of companies use dimensional tagging (AAPL, JPM, JNJ). Others aggregate to PEO vs Non-PEO NEO averages (MSFT, XOM). * * * Pay vs Performance (XBRL - High Reliability) -------------------------------------------- Pay vs Performance disclosures correlate executive compensation with company performance metrics. ### Primary Metrics | Property | Type | XBRL Concept | Description | Example | | --- | --- | --- | --- | --- | | `total_shareholder_return` | `Decimal` | `ecd:TotalShareholderRtnAmt` | Company TSR (cumulative %) | `207.6` | | `peer_group_tsr` | `Decimal` | `ecd:PeerGroupTotalShareholderRtnAmt` | Peer group TSR | `189.3` | | `net_income` | `Decimal` | `us-gaap:NetIncomeLoss` | Net income (USD) | `93,736,000,000` | ### Company-Selected Performance Measure | Property | Type | XBRL Concept | Description | Example | | --- | --- | --- | --- | --- | | `company_selected_measure` | `str` | `ecd:CoSelectedMeasureName` | Company's chosen KPI name | `"Operating Cash Flow"` | | `company_selected_measure_value` | `Decimal` | `ecd:CoSelectedMeasureAmt` | KPI value | `118,254,000,000` | ### Most Important Performance Measures | Property | Type | XBRL Concept | Description | | --- | --- | --- | --- | | `performance_measures` | `list[str]` | `ecd:MeasureName` | List of performance measures used | Example values: `["Revenue", "Operating Income", "Free Cash Flow", "Total Shareholder Return"]` ### Pay vs Performance DataFrame `pvp_df = proxy.pay_vs_performance # pd.DataFrame` | Column | Type | Description | | --- | --- | --- | | `fiscal_year_end` | `date` | End of fiscal year | | `peo_actually_paid_comp` | `Decimal` | CEO compensation actually paid | | `neo_avg_actually_paid_comp` | `Decimal` | NEO average CAP | | `total_shareholder_return` | `Decimal` | Company TSR | | `peer_group_tsr` | `Decimal` | Peer group TSR | | `net_income` | `Decimal` | Net income | | `company_selected_measure_value` | `Decimal` | Company KPI value | * * * Governance Indicators (XBRL) ---------------------------- | Property | Type | XBRL Concept | Description | Example | | --- | --- | --- | --- | --- | | `insider_trading_policy_adopted` | `bool` | `ecd:InsiderTrdPoliciesProcAdoptedFlag` | Has adopted insider trading policy | `True` | * * * XBRL Concept Reference ---------------------- ### Universal Concepts (Present in ALL Companies) These 25 concepts are available across all sampled DEF 14A filings (100% coverage): #### Executive Compensation | Concept | Description | | --- | --- | | `ecd:PeoTotalCompAmt` | PEO total compensation from Summary Compensation Table | | `ecd:PeoActuallyPaidCompAmt` | PEO compensation actually paid | | `ecd:NonPeoNeoAvgTotalCompAmt` | Non-PEO NEO average total compensation | | `ecd:NonPeoNeoAvgCompActuallyPaidAmt` | Non-PEO NEO average compensation actually paid | | `ecd:AdjToCompAmt` | Adjustments to compensation (reconciliation) | | `ecd:PeoName` | Name of Principal Executive Officer | #### Performance Metrics | Concept | Description | | --- | --- | | `ecd:TotalShareholderRtnAmt` | Company total shareholder return | | `ecd:PeerGroupTotalShareholderRtnAmt` | Peer group total shareholder return | | `us-gaap:NetIncomeLoss` | Net income (GAAP) | | `ecd:CoSelectedMeasureAmt` | Company-selected performance measure value | | `ecd:CoSelectedMeasureName` | Company-selected performance measure name | | `ecd:MeasureName` | Names of most important performance measures | #### Text Blocks and Footnotes | Concept | Description | | --- | --- | | `ecd:PvpTableTextBlock` | Pay vs Performance table text block | | `ecd:TabularListTableTextBlock` | Tabular list of performance measures | | `ecd:NamedExecutiveOfficersFnTextBlock` | Named executives footnote | | `ecd:PeerGroupIssuersFnTextBlock` | Peer group issuers footnote | | `ecd:AdjToPeoCompFnTextBlock` | PEO compensation adjustment footnote | | `ecd:AdjToNonPeoNeoCompFnTextBlock` | Non-PEO NEO adjustment footnote | | `ecd:CompActuallyPaidVsTotalShareholderRtnTextBlock` | CAP vs TSR discussion | | `ecd:CompActuallyPaidVsNetIncomeTextBlock` | CAP vs Net Income discussion | | `ecd:CompActuallyPaidVsCoSelectedMeasureTextBlock` | CAP vs company measure discussion | #### Governance | Concept | Description | | --- | --- | | `ecd:InsiderTrdPoliciesProcAdoptedFlag` | Insider trading policy adoption flag | * * * Code Examples ------------- ### Example 1: Extract Executive Compensation `from edgar import Company # Get company and filing company = Company("AAPL") filing = company.get_filings(form="DEF 14A").latest() # Get proxy statement object proxy = filing.obj() # Access executive compensation print(f"CEO: {proxy.peo_name}") print(f"CEO Total Compensation: ${proxy.peo_total_comp:,}") print(f"CEO Compensation Actually Paid: ${proxy.peo_actually_paid_comp:,}") print(f"NEO Average Compensation: ${proxy.neo_avg_actually_paid_comp:,}")` ### Example 2: Pay vs Performance Analysis `from edgar import Company company = Company("MSFT") filing = company.get_filings(form="DEF 14A").latest() proxy = filing.obj() # Get pay vs performance DataFrame pvp = proxy.pay_vs_performance # Calculate pay-for-performance correlation correlation = pvp['peo_actually_paid_comp'].corr(pvp['total_shareholder_return']) print(f"CEO Pay vs TSR Correlation: {correlation:.2f}") # Compare to peer group print(f"Company TSR: {proxy.total_shareholder_return}%") print(f"Peer Group TSR: {proxy.peer_group_tsr}%") print(f"Outperformance: {proxy.total_shareholder_return - proxy.peer_group_tsr:.1f}%")` ### Example 3: Governance Check `from edgar import Company company = Company("JPM") filing = company.get_filings(form="DEF 14A").latest() proxy = filing.obj() # Check governance indicators if proxy.insider_trading_policy_adopted: print("Insider Trading Policy: Adopted") else: print("Insider Trading Policy: Not Adopted (flag for review)") # List performance measures used print("Performance Measures:") for measure in proxy.performance_measures: print(f" - {measure}")` ### Example 4: Multi-Company Comparison `from edgar import Company import pandas as pd tickers = ["AAPL", "MSFT", "GOOGL", "META", "AMZN"] data = [] for ticker in tickers: company = Company(ticker) filing = company.get_filings(form="DEF 14A").latest() proxy = filing.obj() data.append({ 'company': company.name, 'ceo': proxy.peo_name, 'ceo_total_comp': proxy.peo_total_comp, 'ceo_actually_paid': proxy.peo_actually_paid_comp, 'tsr': proxy.total_shareholder_return, 'peer_tsr': proxy.peer_group_tsr }) comparison_df = pd.DataFrame(data) print(comparison_df.to_string())` ### Example 5: Access Board and Director Information The `ProxyStatement` class focuses on XBRL-based executive compensation data. Board composition, director details, and shareholder proposals live in the HTML body of the filing and are not yet extracted into structured properties. However, you can access this information today using the `Filing` object's built-in search and HTML capabilities. `from edgar import Company # Get a DEF 14A filing company = Company("AAPL") filing = company.get_filings(form="DEF 14A").latest() # Search the filing HTML for board-related sections results = filing.search("board of directors") for section in results[:3]: print(section[:200]) # Preview matching sections` `# Search for specific governance topics director_sections = filing.search("director nominees") ownership_sections = filing.search("beneficial ownership") proposal_sections = filing.search("proposal") audit_sections = filing.search("audit fees")` `# Get the full HTML for manual inspection or custom parsing html_content = filing.html() # Or access the filing document directly doc = filing.document()` > **Note**: Board composition, director bios, beneficial ownership tables, and shareholder proposals are available in the filing HTML but require custom parsing. Structured `Director` and `Proposal` objects are planned for a future release. See the [HTML-Based Data](https://edgartools.readthedocs.io/en/latest/guides/proxystatement-data-object-guide/#html-based-data-future-features) > section below for details on what data is available and extraction patterns. * * * View Design Recommendations --------------------------- ### Primary View Components 1. **Header Section** 2. Company name (prominent) 3. Form type with amendment indicator 4. Filing date and fiscal year end 5. Annual meeting date (if available) 6. **Compensation Dashboard** 7. CEO compensation card (Total SCT vs Actually Paid) 8. NEO average compensation card 9. 5-year trend sparkline or chart 10. Year-over-year change indicators 11. **Pay vs Performance Panel** 12. TSR comparison chart (company vs peer group) 13. Compensation vs TSR correlation visualization 14. Net income trend overlay 15. Company-selected performance measure 16. **Governance Indicators** 17. Insider trading policy status badge 18. Performance measures list 19. **Key Metrics Cards** 20. Total Shareholder Return 21. Peer Group TSR 22. Net Income 23. Company KPI with label ### Data Priority for Display | Priority | Data | Reason | | --- | --- | --- | | High | CEO compensation (both SCT and CAP) | Primary user interest | | High | Total Shareholder Return | Key performance metric | | High | Peer group comparison | Benchmark context | | Medium | NEO average compensation | Executive team context | | Medium | Net income | Financial performance | | Medium | Company-selected measure | Company's chosen KPI | | Medium | 5-year compensation trends | Historical context | | Low | Adjustment details | Technical reconciliation | | Low | Footnote text blocks | Reference material | ### Value Formatting | Data Type | Format | Example | | --- | --- | --- | | Compensation | Currency with commas | `$168,980,568` | | Large values (>$1B) | Abbreviated | `$93.7B` | | TSR | Percentage with 1 decimal | `207.6%` | | Year-over-year change | Signed percentage | `+12.5%` or `-8.3%` | ### Visual Indicators (Suggested) | Condition | Visual Treatment | | --- | --- | | Amendment (`/A`) | Yellow "Amendment" badge | | TSR > Peer TSR | Green upward arrow | | TSR < Peer TSR | Red downward arrow | | Insider policy adopted | Green checkmark | | Insider policy not adopted | Red warning icon | | Compensation increase >25% YoY | Orange highlight | | Compensation decrease | Blue highlight | ### Compensation Card Layout `+----------------------------------+ | CEO Compensation | | Mr. Tim Cook | +----------------------------------+ | Summary Comp Table | | $74,609,802 | | +18.0% vs prior year | +----------------------------------+ | Compensation Actually Paid | | $168,980,568 | | +58.4% vs prior year | +----------------------------------+` * * * Example Data Structure ---------------------- `{ # Metadata "form": "DEF 14A", "filing_date": "2025-01-10", "fiscal_year_end": "2024-09-28", "company_name": "Apple Inc.", "cik": "0000320193", "accession_number": "0001308179-25-000008", # Executive Compensation "peo_name": "Mr. Cook", "peo_total_comp": 74609802, "peo_actually_paid_comp": 168980568, "neo_avg_total_comp": 27178896, "neo_avg_actually_paid_comp": 58633525, # Pay vs Performance "total_shareholder_return": 207.6, "peer_group_tsr": 189.3, "net_income": 93736000000, "company_selected_measure": "Operating Cash Flow", "company_selected_measure_value": 118254000000, # Performance Measures "performance_measures": [ "Net Sales", "Operating Income", "Total Shareholder Return", "Operating Cash Flow" ], # Governance "insider_trading_policy_adopted": True, # Named Executives (when dimensional data available) "has_individual_executive_data": True, "named_executives": [ { "id": "aapl:CookMember", "name": "Mr. Cook", "role": "PEO", "total_comp": 74609802, "actually_paid_comp": 168980568 }, { "id": "aapl:MaestriMember", "name": "Luca Maestri", "role": "NEO", "total_comp": 27178896, "actually_paid_comp": 58633525 } ], # Time Series (5 years) "executive_compensation": [ { "fiscal_year_end": "2024-09-28", "peo_total_comp": 74609802, "peo_actually_paid_comp": 168980568, "neo_avg_total_comp": 27178896, "neo_avg_actually_paid_comp": 58633525 }, { "fiscal_year_end": "2023-09-30", "peo_total_comp": 63209845, "peo_actually_paid_comp": 106643588, "neo_avg_total_comp": 26938240, "neo_avg_actually_paid_comp": 48892163 } # ... 3 more years ], # Pay vs Performance Series "pay_vs_performance": [ { "fiscal_year_end": "2024-09-28", "peo_actually_paid_comp": 168980568, "neo_avg_actually_paid_comp": 58633525, "total_shareholder_return": 207.6, "peer_group_tsr": 189.3, "net_income": 93736000000 } # ... more years ] }` * * * HTML-Based Data (Future Features) --------------------------------- The following data is available in DEF 14A HTML sections but not yet extracted into structured properties. You can access these sections today using `filing.search()` to find relevant content. ### Beneficial Ownership | Section | Description | | --- | --- | | Principal Shareholders | Shareholders owning >5% of shares | | Director/Executive Ownership | Shares owned by insiders | **How to access today**: `results = filing.search("beneficial ownership") # or results = filing.search("security ownership")` ### Board of Directors | Data | Description | | --- | --- | | Director Names | Full list of board members | | Director Ages | Ages of directors | | Director Tenure | Years on board | | Independence Status | Independent vs non-independent | | Committee Memberships | Audit, Compensation, Governance | **How to access today**: `results = filing.search("board of directors") # or results = filing.search("director nominees")` ### Director Compensation | Data | Description | | --- | --- | | Director Fees | Annual retainer and meeting fees | | Stock Awards | Equity compensation | | Total Compensation | Sum of all compensation | **How to access today**: `results = filing.search("director compensation")` ### Voting Proposals | Proposal Type | Description | | --- | --- | | Election of Directors | Board member elections | | Ratification of Auditors | Audit firm approval | | Say-on-Pay | Executive compensation advisory vote | | Shareholder Proposals | Proposals submitted by shareholders | | Equity Plan Amendments | Stock compensation plan changes | **How to access today**: `results = filing.search("proposal")` ### Audit Information | Data | Description | | --- | --- | | Auditor Name | Independent auditor firm | | Audit Fees | Fees for audit services | | Tax Fees | Fees for tax services | | Other Fees | Other professional fees | **How to access today**: `results = filing.search("audit fees")` * * * Notes for Implementation ------------------------ 1. **XBRL Namespace**: The primary namespace for proxy data is `ecd:` (Executive Compensation Disclosure), introduced by SEC in 2022 for fiscal years ending on or after December 16, 2022. 2. **Dimensional Tagging Variation**: 3. ~60% of companies tag individual executive data using `dim_ecd_IndividualAxis` 4. ~40% only provide aggregate data (PEO and Non-PEO NEO averages) 5. Always check `has_individual_executive_data` before accessing individual executives 6. **Time Series Data**: Pay vs Performance tables include 5 years of historical data per SEC requirements. This enables robust trend analysis. 7. **Compensation Actually Paid (CAP)**: This SEC-mandated metric differs from Summary Compensation Table totals due to adjustments for: 8. Change in pension value 9. Stock/option awards fair value changes 10. Vesting date fair values 11. **Company-Selected Measure**: Each company chooses one performance measure they consider most important. Common choices include: 12. Revenue or Net Sales 13. Operating Income 14. Free Cash Flow 15. Adjusted EBITDA 16. Return on Invested Capital 17. **Peer Group**: Companies define their own peer groups for TSR comparison. The peer group composition is disclosed in the filing but not structured in XBRL. 18. **Amendment Handling**: DEF 14A/A filings contain corrections or updates. Always use the most recent filing for a given fiscal year. 19. **Form Variants**: 20. `DEF 14A`: Standard proxy statement 21. `DEFA14A`: Additional soliciting materials (may have limited data) 22. `DEFM14A`: Merger-related proxy (may have different structure) 23. **Fiscal Year Alignment**: Match compensation periods with company fiscal year ends, which vary by company (e.g., Apple uses September, Microsoft uses June). 24. **Large Cap Coverage**: XBRL data is highly reliable for S&P 500 companies. Smaller companies may have less complete tagging. Back to top --- # ABS Distribution (10-D) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/tend-data-object-guide/#form-10-d-abs-distribution-reports-cmbs) Form 10-D: ABS Distribution Reports (CMBS) ========================================== Overview -------- **Form 10-D** is an Asset-Backed Issuer Distribution Report required under Sections 13 and 15(d) of the Securities Exchange Act of 1934. These filings disclose distribution and pool performance data for publicly offered asset-backed securities (ABS). ABS come in many forms — credit card receivables, auto loans, student loans, residential mortgages (RMBS), commercial mortgages (CMBS), and utility securitizations. However, **edgartools currently supports only CMBS (Commercial Mortgage-Backed Securities) filings** that include structured XML asset data in the EX-102 attachment. For CMBS 10-D filings, edgartools extracts: - **Loan-level data**: 49 fields covering origination, balance, rate, maturity, servicer, and payment status - **Property-level data**: 46 fields covering location, type, valuation, occupancy, financials, and debt service coverage ratios - **Summary statistics**: Pool metrics like total balance, average DSCR, occupancy, and property type distribution ### What About Non-CMBS 10-D Filings? For non-CMBS 10-D filings (auto loans, credit cards, student loans, etc.), `filing.obj()` returns `None` because these filings do not include standardized XML asset data. Asset data for non-CMBS filings is typically reported in separate ABS-EE filings or in HTML tables with highly variable formats. Access Pattern -------------- `from edgar import Filing # Get a Form 10-D filing filing = Filing(form="10-D", ...) # Parse into TenD object ten_d = filing.obj() # Returns TenD for CMBS, None for non-CMBS # Check if CMBS data is available if ten_d and ten_d.has_asset_data: loans = ten_d.loans # pandas DataFrame properties = ten_d.properties # pandas DataFrame summary = ten_d.asset_data.summary() # CMBSSummary` * * * Display ------- When you call `filing.obj()` on a CMBS Form 10-D filing, edgartools parses the filing and displays a rich panel with issuer information, distribution period, and asset data summary: ![TenD Display](https://edgartools.readthedocs.io/en/stable/images/tend-display.webp) The panel shows: - Issuing entity (ABS trust name and CIK) - ABS type (detected as CMBS, AUTO, CREDIT\_CARD, etc.) - Distribution period dates - Filing date - Depositor and sponsor entities - Security classes registered - Asset data summary (loans, properties, total balance, average rates/DSCR/occupancy, property types) * * * Core Data Structure ------------------- ### TenD (Top-Level Object) | Property | Type | Description | | --- | --- | --- | | `filing` | `Filing` | The underlying Filing object | | `form` | `str` | Form type (10-D or 10-D/A for amendments) | | `company` | `str` | Company name from the filing | | `filing_date` | `date` | Filing date | | `accession_number` | `str` | SEC accession number | | `issuing_entity` | `ABSEntity` | The ABS trust/entity (parsed from HTML header) | | `depositor` | `ABSEntity` | The depositor entity | | `sponsors` | `List[ABSEntity]` | Sponsor entities | | `distribution_period` | `DistributionPeriod` | Period covered (start\_date, end\_date) | | `security_classes` | `List[str]` | Classes of securities registered | | `abs_type` | `ABSType` | Detected ABS type enum (CMBS, AUTO, CREDIT\_CARD, RMBS, STUDENT\_LOAN, UTILITY, OTHER) | | `has_asset_data` | `bool` | Whether EX-102 XML exists (CMBS indicator) | | `asset_data` | `Optional[CMBSAssetData]` | Parsed CMBS asset data (lazy-loaded) | | `loans` | `Optional[DataFrame]` | Convenience property for `asset_data.loans` | | `properties` | `Optional[DataFrame]` | Convenience property for `asset_data.properties` | * * * Supporting Data Models ---------------------- ### ABSEntity Represents an entity involved in the ABS transaction (issuer, depositor, or sponsor). | Field | Type | Description | | --- | --- | --- | | `name` | `str` | Entity name | | `cik` | `Optional[str]` | SEC Central Index Key | | `file_number` | `Optional[str]` | Commission file number | ### DistributionPeriod The distribution period covered by the filing. | Field | Type | Description | | --- | --- | --- | | `start_date` | `Optional[date]` | Period start date | | `end_date` | `Optional[date]` | Period end date | ### ABSType (Enum) Detected asset-backed security type. EdgarTools detects this by looking for EX-102 XML (CMBS) or by analyzing company/issuer name keywords. | Value | Description | | --- | --- | | `CMBS` | Commercial Mortgage-Backed Securities | | `AUTO` | Auto Loan/Lease ABS | | `CREDIT_CARD` | Credit Card Receivables | | `RMBS` | Residential Mortgage-Backed Securities | | `STUDENT_LOAN` | Student Loan ABS | | `UTILITY` | Utility Securitizations | | `OTHER` | Other asset types | * * * CMBS Asset Data --------------- ### CMBSAssetData The `asset_data` property provides access to the structured CMBS loan and property data parsed from the EX-102 XML attachment. | Property | Type | Description | | --- | --- | --- | | `loans` | `DataFrame` | Loan-level data (49 fields) | | `properties` | DataFrame\` | Property-level data (46 fields) | | `summary()` | `CMBSSummary` | Aggregate statistics for the pool | ### Loans DataFrame (49 Fields) The `loans` DataFrame contains one row per loan with the following key columns: | Column | Type | Description | | --- | --- | --- | | `loan_id` | `str` | Prospectus loan identifier | | `originator` | `str` | Loan originator name | | `origination_date` | `date` | Date loan was originated | | `original_amount` | `float` | Original loan amount | | `original_term_months` | `int` | Original loan term in months | | `maturity_date` | `date` | Loan maturity date | | `original_rate` | `float` | Original interest rate | | `current_rate` | `float` | Current interest rate | | `actual_balance` | `float` | Current actual balance | | `scheduled_balance` | `float` | Scheduled principal balance at securitization | | `payment_status` | `str` | Payment status code (0=current, 1=30 days, etc.) | | `is_modified` | `bool` | Whether loan has been modified | | `is_balloon` | `bool` | Whether loan has balloon payment | | `is_interest_only` | `bool` | Whether loan is interest-only | | `primary_servicer` | `str` | Primary servicer name | | `lien_position` | `str` | Lien position code | | `num_properties` | `int` | Number of properties securing the loan | **Additional loan fields** include: `loan_id_type`, `period_start`, `period_end`, `securitization_rate`, `accrual_method`, `rate_type`, `io_term_months`, `first_payment_date`, `loan_structure`, `payment_type`, `payment_frequency`, `num_properties_securitization`, `grace_days`, `has_prepayment_premium`, `has_negative_amortization`, `lockout_end_date`, `yield_maintenance_end_date`, `prepayment_premium_end_date`, `period_begin_balance`, `scheduled_pi_due`, `servicer_fee_rate`, `scheduled_interest`, `scheduled_principal`, `unscheduled_principal`, `scheduled_end_balance`, `paid_through_date`, `servicing_advance_method`, `pi_advances_outstanding`, `ti_advances_outstanding`, `other_advances_outstanding`, `subject_to_demand`. ### Properties DataFrame (46 Fields) The `properties` DataFrame contains one or more rows per loan (one per property securing the loan) with the following key columns: | Column | Type | Description | | --- | --- | --- | | `loan_id` | `str` | Associated loan identifier | | `name` | `str` | Property name | | `address` | `str` | Street address | | `city` | `str` | City | | `state` | `str` | State code | | `zip` | `str` | ZIP code | | `county` | `str` | County | | `property_type` | `str` | Property type code (MF, OF, RT, etc.) | | `units` | `int` | Number of units/beds/rooms | | `sqft` | `int` | Net rentable square feet | | `year_built` | `int` | Year property was built | | `year_renovated` | `int` | Year last renovated | | `valuation` | `float` | Property valuation amount | | `valuation_source` | `str` | Valuation source code | | `valuation_date` | `date` | Valuation date | | `occupancy_securitization` | `float` | Occupancy % at securitization | | `occupancy_current` | `float` | Most recent occupancy % | | `revenue_securitization` | `float` | Revenue at securitization | | `opex_securitization` | `float` | Operating expenses at securitization | | `noi_securitization` | `float` | Net Operating Income at securitization | | `ncf_securitization` | `float` | Net Cash Flow at securitization | | `dscr_noi_securitization` | `float` | Debt Service Coverage Ratio (NOI) at securitization | | `dscr_ncf_securitization` | `float` | Debt Service Coverage Ratio (NCF) at securitization | **Additional property fields** include: `units_securitization`, `sqft_securitization`, `status`, `defeased_status`, `tenant_1_name`, `tenant_1_sqft`, `tenant_1_lease_exp`, `tenant_2_name`, `tenant_2_sqft`, `tenant_2_lease_exp`, `tenant_3_name`, `tenant_3_sqft`, `tenant_3_lease_exp`, `financials_date_securitization`, `financials_start_date`, `financials_end_date`, `revenue_current`, `opex_current`, `noi_current`, `ncf_current`, `debt_service_current`, `dscr_noi_current`, `dscr_ncf_current`. ### CMBSSummary The `summary()` method aggregates the loan and property data into pool-level statistics. | Field | Type | Description | | --- | --- | --- | | `num_loans` | `int` | Total number of loans | | `num_properties` | `int` | Total number of properties | | `total_loan_balance` | `float` | Sum of actual balances | | `total_original_loan_amount` | `float` | Sum of original amounts | | `avg_interest_rate` | `Optional[float]` | Average current interest rate | | `avg_dscr` | `Optional[float]` | Average DSCR (NOI-based) | | `avg_occupancy` | `Optional[float]` | Average occupancy % | | `property_types` | `Dict[str, int]` | Property type code → count | | `states` | `Dict[str, int]` | State → count | | `delinquent_loans` | `int` | Number of loans with payment\_status != '0' | | `modified_loans` | `int` | Number of modified loans | * * * Property Type Codes ------------------- Common values for `property_type` in the properties DataFrame: | Code | Description | | --- | --- | | `MF` | Multifamily | | `OF` | Office | | `RT` | Retail | | `IN` | Industrial | | `LO` | Lodging/Hotel | | `HC` | Healthcare | | `SS` | Self Storage | | `MH` | Manufactured Housing | | `OT` | Other | * * * Example: Analyzing a CMBS Pool ------------------------------ `from edgar import find # Find a CMBS 10-D filing (BANK5 2024-5YR9) filing = find('0001888524-25-020550') ten_d = filing.obj() # Display the filing print(ten_d) # Shows the rich panel with issuer, distribution period, and asset data summary # Check if CMBS data is available print(ten_d.has_asset_data) # True print(ten_d.abs_type) # ABSType.CMBS # Access loan data loans = ten_d.loans print(f"Number of loans: {len(loans)}") print(loans[['loan_id', 'actual_balance', 'current_rate', 'payment_status']].head()) # Access property data properties = ten_d.properties print(f"Number of properties: {len(properties)}") print(properties[['name', 'city', 'state', 'property_type', 'valuation']].head()) # Get pool summary summary = ten_d.asset_data.summary() print(f"Total balance: ${summary.total_loan_balance:,.0f}") print(f"Average rate: {summary.avg_interest_rate:.2%}") print(f"Average DSCR: {summary.avg_dscr:.2f}") print(f"Delinquent loans: {summary.delinquent_loans}") # Property type distribution for prop_type, count in sorted(summary.property_types.items(), key=lambda x: -x[1]): print(f" {prop_type}: {count}") # Top states by property count for state, count in sorted(summary.states.items(), key=lambda x: -x[1])[:5]: print(f" {state}: {count}")` * * * Example: Finding High-DSCR Properties ------------------------------------- `from edgar import find filing = find('0001888524-25-020550') ten_d = filing.obj() # Filter properties with strong debt service coverage properties = ten_d.properties high_dscr = properties[properties['dscr_noi_securitization'] > 1.5] print(f"Found {len(high_dscr)} properties with DSCR > 1.5") print(high_dscr[['name', 'property_type', 'state', 'dscr_noi_securitization']].sort_values('dscr_noi_securitization', ascending=False))` * * * Example: Tracking Delinquencies ------------------------------- `from edgar import find filing = find('0001888524-25-020550') ten_d = filing.obj() # Find delinquent loans (payment_status != '0') loans = ten_d.loans delinquent = loans[loans['payment_status'] != '0'] print(f"Delinquent loans: {len(delinquent)} out of {len(loans)}") if len(delinquent) > 0: print(delinquent[['loan_id', 'actual_balance', 'payment_status', 'primary_servicer']])` * * * Example: Geographic Distribution -------------------------------- `from edgar import find import pandas as pd filing = find('0001888524-25-020550') ten_d = filing.obj() # Aggregate property value by state properties = ten_d.properties state_totals = properties.groupby('state').agg({ 'valuation': 'sum', 'loan_id': 'count' }).rename(columns={'loan_id': 'num_properties'}) state_totals = state_totals.sort_values('valuation', ascending=False) print(state_totals.head(10))` * * * Searching for 10-D Filings -------------------------- `from edgar import get_filings, Company # Search for all recent 10-D filings recent_10d = get_filings(form="10-D").head(20) for filing in recent_10d: ten_d = filing.obj() # Only CMBS filings will have asset_data if ten_d and ten_d.has_asset_data: print(f"{ten_d.issuing_entity.name}: {ten_d.abs_type.value}") summary = ten_d.asset_data.summary() print(f" {summary.num_loans} loans, ${summary.total_loan_balance:,.0f}") # Search for 10-D filings by a specific company company = Company("1888524") # BANK as Depositor filings = company.get_filings(form="10-D")` * * * Important Notes --------------- 1. **CMBS-only support**: `filing.obj()` returns `TenD` only for CMBS filings with EX-102 XML asset data. Non-CMBS 10-D filings return `None`. 2. **DataFrames are pandas DataFrames**: You can use standard pandas operations to filter, aggregate, and analyze the data. 3. **Lazy loading**: The XML asset data is parsed on first access to `asset_data`, `loans`, or `properties`. 4. **Missing data**: Many fields can be `None` or `NaN`. Always handle missing data gracefully when doing calculations. 5. **Distribution report HTML parsing is not supported**: The narrative distribution report section has highly variable HTML formats across issuers and was found to have only ~42% extraction accuracy. EdgarTools focuses on the structured EX-102 XML data. * * * Data Source ----------- The structured CMBS data comes from the **EX-102 XML attachment** filed with the 10-D. The XML follows the SEC EDGAR schema: `http://www.sec.gov/edgar/document/absee/cmbs/assetdata` The issuer, depositor, sponsor, distribution period, and security class information is parsed from the HTML header of the 10-D filing. * * * Common Use Cases ---------------- ### Portfolio Risk Analysis * Identify loans with low DSCR or high vacancy * Track delinquency rates across servicers * Monitor properties in specific states or metros ### Investment Research * Compare pool composition across issuers * Analyze property type concentrations * Evaluate geographic diversification ### Compliance Monitoring * Track modified loans * Monitor payment status trends * Review servicer performance ### Market Intelligence * Analyze securitization activity by sponsor * Track originator market share * Compare interest rates across pools Back to top --- # Crowdfunding (Form C) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/formc-data-object-guide/#form-c-parse-crowdfunding-offerings-regulation-cf) Form C: Parse Crowdfunding Offerings (Regulation CF) ==================================================== Overview -------- **Form C** is an SEC filing used by companies raising capital through crowdfunding under Regulation Crowdfunding (Regulation CF). These filings provide insights into small business fundraising campaigns, including offering terms, issuer financials, and funding portal information. The `FormC` class in edgartools parses Form C XML filings into structured Python objects, making it easy to track crowdfunding campaigns, analyze issuer financials, and monitor offering progress. What Form C Covers ------------------ Form C encompasses several filing types that track the complete lifecycle of a crowdfunding offering: * **Form C** - Initial crowdfunding offering * **Form C/A** - Amendment to the offering * **Form C-U** - Progress update during the offering period * **Form C-AR** - Annual report (filed yearly after offering closes) * **Form C-TR** - Termination report (when offering is withdrawn) Access Pattern -------------- `from edgar import get_filings # Get recent Form C filings filing = get_filings(form="C").head(1)[0] # Parse into FormC object formc = filing.obj() # Access offering details print(f"{formc.issuer_name}") print(f"Target: ${formc.offering_information.target_amount:,.0f}") print(f"Maximum: ${formc.offering_information.maximum_offering_amount:,.0f}") print(f"Deadline: {formc.offering_information.deadline_date}")` When you call `filing.obj()` on a Form C filing, edgartools displays a rich formatted panel showing all filing details: ![Form C Display](https://edgartools.readthedocs.io/en/stable/images/formc-display.webp) * * * Core Data Structure ------------------- ### FormC (Top-Level Object) | Property | Type | Description | | --- | --- | --- | | `filer_information` | `FilerInformation` | CIK, filing period, test/live flag | | `issuer_information` | `IssuerInformation` | Company name, address, website, legal status, jurisdiction | | `offering_information` | `Optional[OfferingInformation]` | Offering terms (present in C, C-U; absent in C-AR, C-TR) | | `annual_report_disclosure` | `Optional[AnnualReportDisclosure]` | Financial disclosures (present in C-AR) | | `signature_info` | `SignatureInfo` | Officer and director signatures | | `form` | `str` | Form type (C, C/A, C-U, C-AR, C-TR) | | `description` | `str` | Human-readable form description | | `issuer_name` | `str` | Convenience property for issuer name | | `portal_name` | `Optional[str]` | Funding portal name (if present) | | `campaign_status` | `str` | Derived status (Active, Terminated, Annual Report, etc.) | | `days_to_deadline` | `Optional[int]` | Days until offering deadline (negative if expired) | | `is_expired` | `bool` | Whether offering deadline has passed | ### Computed Properties | Property | Returns | Description | | --- | --- | --- | | `issuer` | `IssuerCompany` | Company wrapper with offering-specific methods | | `docs` | `Docs` | Access comprehensive API documentation | ### Methods | Method | Returns | Description | | --- | --- | --- | | `get_offering()` | `Offering` | Complete offering lifecycle (all related filings) | | `to_context(detail)` | `str` | Token-efficient text representation for AI context | * * * Offering Information -------------------- The `offering_information` property contains the core fundraising terms. It is present in Form C and Form C-U but **not** in Form C-AR or C-TR. ### OfferingInformation | Property | Type | Description | | --- | --- | --- | | `compensation_amount` | `str` | Portal fees and compensation structure | | `security_offered_type` | `Optional[str]` | Type of security (Equity, Debt, Other) | | `security_offered_other_desc` | `Optional[str]` | Description if type is "Other" | | `offering_amount` | `Optional[float]` | Target offering amount in dollars | | `maximum_offering_amount` | `Optional[float]` | Maximum offering amount (for over-subscription) | | `price` | `Optional[str]` | Price per security (as string) | | `no_of_security_offered` | `Optional[str]` | Number of securities offered (as string) | | `deadline_date` | `Optional[date]` | Offering deadline | | `over_subscription_accepted` | `Optional[str]` | "Y" or "N" | | `over_subscription_allocation_type` | `Optional[str]` | How over-subscriptions are allocated | | `price_determination_method` | `Optional[str]` | How price was determined | ### Computed Properties | Property | Returns | Description | | --- | --- | --- | | `security_description` | `str` | Combined type and description | | `target_amount` | `Optional[float]` | Alias for `offering_amount` | | `offering_deadline` | `Optional[date]` | Alias for `deadline_date` | | `price_per_security` | `Optional[float]` | Price parsed as float | | `number_of_securities` | `Optional[int]` | Number of securities as int | | `percent_to_maximum` | `Optional[float]` | Target as percentage of maximum | ### Example: Analyzing Offering Terms `from edgar import get_filings filing = get_filings(form="C").head(1)[0] formc = filing.obj() offering = formc.offering_information if offering: print(f"Security: {offering.security_description}") print(f"Target: ${offering.target_amount:,.0f}") print(f"Maximum: ${offering.maximum_offering_amount:,.0f}") print(f"Target is {offering.percent_to_maximum:.0f}% of maximum") if offering.price_per_security and offering.number_of_securities: print(f"Price: ${offering.price_per_security:.2f} per unit") print(f"Units: {offering.number_of_securities:,}") # Check deadline days_left = formc.days_to_deadline if days_left is not None: if days_left > 0: print(f"Deadline: {days_left} days remaining") else: print(f"Offering expired {abs(days_left)} days ago")` * * * Financial Disclosures --------------------- The `annual_report_disclosure` property contains financial statements for the current and prior fiscal year. It is present in **Form C-AR** (annual reports) and sometimes in Form C. ### AnnualReportDisclosure | Property | Type | Description | | --- | --- | --- | | `current_employees` | `int` | Number of employees | | `total_asset_most_recent_fiscal_year` | `float` | Total assets (current year) | | `total_asset_prior_fiscal_year` | `float` | Total assets (prior year) | | `cash_equi_most_recent_fiscal_year` | `float` | Cash and equivalents (current) | | `cash_equi_prior_fiscal_year` | `float` | Cash and equivalents (prior) | | `act_received_most_recent_fiscal_year` | `float` | Accounts receivable (current) | | `act_received_prior_fiscal_year` | `float` | Accounts receivable (prior) | | `short_term_debt_most_recent_fiscal_year` | `float` | Short-term debt (current) | | `short_term_debt_prior_fiscal_year` | `float` | Short-term debt (prior) | | `long_term_debt_most_recent_fiscal_year` | `float` | Long-term debt (current) | | `long_term_debt_prior_fiscal_year` | `float` | Long-term debt (prior) | | `revenue_most_recent_fiscal_year` | `float` | Revenue (current) | | `revenue_prior_fiscal_year` | `float` | Revenue (prior) | | `cost_goods_sold_most_recent_fiscal_year` | `float` | Cost of goods sold (current) | | `cost_goods_sold_prior_fiscal_year` | `float` | Cost of goods sold (prior) | | `tax_paid_most_recent_fiscal_year` | `float` | Taxes paid (current) | | `tax_paid_prior_fiscal_year` | `float` | Taxes paid (prior) | | `net_income_most_recent_fiscal_year` | `float` | Net income (current) | | `net_income_prior_fiscal_year` | `float` | Net income (prior) | | `offering_jurisdictions` | `List[str]` | States where offering is available | ### Computed Financial Metrics | Property | Returns | Description | | --- | --- | --- | | `total_debt_most_recent` | `float` | Short-term + long-term debt (current year) | | `total_debt_prior` | `float` | Short-term + long-term debt (prior year) | | `debt_to_asset_ratio` | `Optional[float]` | Debt-to-asset ratio as percentage | | `revenue_growth_yoy` | `Optional[float]` | Year-over-year revenue growth percentage | | `asset_growth_yoy` | `Optional[float]` | Year-over-year asset growth percentage | | `is_pre_revenue` | `bool` | True if no revenue in current year | | `burn_rate_change` | `Optional[float]` | Change in net income (shows burn rate trend) | ### Convenience Aliases (Most Recent Year) For easier access to current year data: | Property | Returns | Maps To | | --- | --- | --- | | `total_assets` | `float` | `total_asset_most_recent_fiscal_year` | | `cash_and_cash_equivalents` | `float` | `cash_equi_most_recent_fiscal_year` | | `accounts_receivable` | `float` | `act_received_most_recent_fiscal_year` | | `short_term_debt` | `float` | `short_term_debt_most_recent_fiscal_year` | | `long_term_debt` | `float` | `long_term_debt_most_recent_fiscal_year` | | `revenues` | `float` | `revenue_most_recent_fiscal_year` | | `cost_of_goods_sold` | `float` | `cost_goods_sold_most_recent_fiscal_year` | | `taxes_paid` | `float` | `tax_paid_most_recent_fiscal_year` | | `net_income` | `float` | `net_income_most_recent_fiscal_year` | | `number_of_employees` | `int` | `current_employees` | ### Example: Analyzing Issuer Financials `from edgar import get_filings # Get a Form C-AR (annual report) filing = get_filings(form="C-AR").head(1)[0] formc_ar = filing.obj() financials = formc_ar.annual_report_disclosure if financials: # Basic metrics print(f"Employees: {financials.number_of_employees}") print(f"Total Assets: ${financials.total_assets:,.0f}") print(f"Cash: ${financials.cash_and_cash_equivalents:,.0f}") # Revenue analysis if financials.is_pre_revenue: print("Status: Pre-revenue") else: print(f"Revenue: ${financials.revenues:,.0f}") if financials.revenue_growth_yoy is not None: print(f"Revenue Growth: {financials.revenue_growth_yoy:+.1f}% YoY") # Profitability print(f"Net Income: ${financials.net_income:,.0f}") # Debt analysis total_debt = financials.total_debt_most_recent if total_debt > 0: print(f"Total Debt: ${total_debt:,.0f}") if financials.debt_to_asset_ratio: print(f"Debt-to-Asset Ratio: {financials.debt_to_asset_ratio:.0f}%")` * * * Issuer Information ------------------ ### IssuerInformation The `issuer_information` property contains company details. | Property | Type | Description | | --- | --- | --- | | `name` | `str` | Legal name of the issuing company | | `address` | `Address` | Business address | | `website` | `str` | Company website | | `co_issuer` | `bool` | Whether there is a co-issuer | | `funding_portal` | `Optional[FundingPortal]` | Crowdfunding portal intermediary | | `legal_status` | `str` | Legal structure (Corporation, LLC, etc.) | | `jurisdiction` | `str` | State/country of incorporation | | `date_of_incorporation` | `date` | Date company was formed | ### FundingPortal The intermediary platform facilitating the crowdfunding raise. | Property | Type | Description | | --- | --- | --- | | `name` | `str` | Portal name (e.g., "StartEngine", "Wefunder") | | `cik` | `str` | SEC Central Index Key for the portal | | `crd` | `Optional[str]` | FINRA CRD number | | `file_number` | `str` | SEC commission file number for the portal | ### IssuerCompany The `issuer` property returns an `IssuerCompany` object that provides offering-specific methods: | Method | Returns | Description | | --- | --- | --- | | `as_company()` | `Company` | Convert to full Company object | | `get_offerings()` | `List[Offering]` | All crowdfunding offerings by this issuer | | `latest_offering()` | `Optional[Offering]` | Most recent offering | ### Example: Working with Issuers `from edgar import get_filings filing = get_filings(form="C").head(1)[0] formc = filing.obj() # Basic issuer info print(f"Company: {formc.issuer_name}") print(f"Incorporated: {formc.issuer_information.jurisdiction} " f"({formc.issuer_information.date_of_incorporation.year})") print(f"Website: {formc.issuer_information.website}") # Funding portal if formc.portal_name: print(f"Portal: {formc.portal_name}") # Get all offerings by this company issuer = formc.issuer offerings = issuer.get_offerings() print(f"{issuer.name} has {len(offerings)} crowdfunding offerings") # Convert to full Company object for more data company = issuer.as_company() print(f"All SEC filings: {len(company.get_filings())}")` * * * Offering Lifecycle ------------------ Each crowdfunding campaign goes through multiple filing stages. The `get_offering()` method returns an `Offering` object that tracks the complete lifecycle. ### Example: Tracking an Offering Over Time `from edgar import get_filings # Get a Form C filing filing = get_filings(form="C").head(1)[0] formc = filing.obj() # Get the complete offering lifecycle offering = formc.get_offering() # View all filings for this offering print(offering.timeline()) # Access specific filing types initial_filing = offering.initial() # Original Form C updates = offering.updates() # Form C-U progress updates annual_reports = offering.annual_reports() # Form C-AR reports amendments = offering.amendments() # Form C/A amendments` * * * Form Variants and Data Availability ----------------------------------- Different Form C variants contain different data sections: | Form | `offering_information` | `annual_report_disclosure` | `funding_portal` | | --- | --- | --- | --- | | **C** | Yes | Sometimes | Yes | | **C/A** | Yes | Sometimes | Yes | | **C-U** | Limited | Sometimes | Yes | | **C-AR** | No | Yes | No | | **C-TR** | No | No | Usually minimal | **Key patterns:** * **Form C** (initial offering): Has full offering terms and portal info * **Form C-U** (progress update): May have limited offering info updates * **Form C-AR** (annual report): Has financial disclosures, no offering terms * **Form C-TR** (termination): Minimal data, indicates offering withdrawn Always check for `None` before accessing `offering_information` or `annual_report_disclosure`: `if formc.offering_information: print(f"Target: ${formc.offering_information.target_amount:,.0f}") if formc.annual_report_disclosure: print(f"Revenue: ${formc.annual_report_disclosure.revenues:,.0f}")` * * * Signature Information --------------------- ### SignatureInfo | Property | Type | Description | | --- | --- | --- | | `issuer_signature` | `IssuerSignature` | Company signature | | `signatures` | `List[PersonSignature]` | Individual officer/director signatures | | `signers` | `List[Signer]` | Consolidated list of unique signers | ### IssuerSignature | Property | Type | Description | | --- | --- | --- | | `issuer` | `str` | Company name as signed | | `title` | `str` | Title of signer (e.g., "Chief Executive Officer") | | `signature` | `str` | Signature as signed | ### PersonSignature | Property | Type | Description | | --- | --- | --- | | `signature` | `str` | Person's name as signed | | `title` | `str` | Title (e.g., "Director") | | `date` | `date` | Date signed | * * * Common Use Cases ---------------- ### Finding Active Offerings `from edgar import get_filings # Get recent Form C filings active_offerings = get_filings(form="C").head(20) for filing in active_offerings: formc = filing.obj() # Filter for non-expired offerings if not formc.is_expired and formc.offering_information: offering = formc.offering_information print(f"{formc.issuer_name}") print(f" Target: ${offering.target_amount:,.0f}") print(f" Deadline: {offering.deadline_date}") print(f" Days left: {formc.days_to_deadline}") print()` ### Analyzing Issuer Financial Health `from edgar import get_filings # Get Form C-AR annual reports annual_reports = get_filings(form="C-AR").head(10) for filing in annual_reports: formc = filing.obj() fin = formc.annual_report_disclosure if fin: print(f"{formc.issuer_name}") print(f" Assets: ${fin.total_assets:,.0f}") print(f" Cash: ${fin.cash_and_cash_equivalents:,.0f}") if fin.is_pre_revenue: print(f" Status: Pre-revenue") else: print(f" Revenue: ${fin.revenues:,.0f}") if fin.revenue_growth_yoy: print(f" Growth: {fin.revenue_growth_yoy:+.1f}% YoY") print(f" Net Income: ${fin.net_income:,.0f}") print()` ### Tracking a Company's Crowdfunding History `from edgar import Company # Get a company that has done crowdfunding company = Company("1881570") # Example: ViiT Health # Get all crowdfunding filings cf_filings = company.get_filings(form=['C', 'C/A', 'C-U', 'C-AR', 'C-TR']) # Group by offering from edgar.offerings.formc import group_offerings_by_file_number grouped = group_offerings_by_file_number(cf_filings) print(f"{company.name} has {len(grouped)} crowdfunding offerings") for file_num, offering_filings in grouped.items(): print(f"\nOffering {file_num}:") for filing in offering_filings: print(f" {filing.form:8} - {filing.filing_date}")` * * * AI-Optimized Context -------------------- The `to_context()` method provides a token-efficient text representation optimized for AI/LLM context windows: `from edgar import get_filings filing = get_filings(form="C").head(1)[0] formc = filing.obj() # Minimal detail (~100-200 tokens) print(formc.to_context(detail='minimal')) # Standard detail (~300-500 tokens) print(formc.to_context(detail='standard')) # Full detail (~600-1000 tokens) print(formc.to_context(detail='full'))` This is useful when passing Form C data to language models or building AI-powered analysis tools. * * * Quick Reference --------------- ### Properties Quick Lookup | Access | Returns | Example | | --- | --- | --- | | `formc.issuer_name` | `str` | Company name | | `formc.portal_name` | `Optional[str]` | Funding portal | | `formc.campaign_status` | `str` | "Active", "Terminated", etc. | | `formc.days_to_deadline` | `Optional[int]` | Days remaining (negative if expired) | | `formc.is_expired` | `bool` | Deadline passed? | | `formc.offering_information` | `Optional[OfferingInformation]` | Offering terms | | `formc.annual_report_disclosure` | `Optional[AnnualReportDisclosure]` | Financial data | | `formc.issuer` | `IssuerCompany` | Company wrapper with offering methods | ### Methods Quick Lookup | Method | Returns | Use Case | | --- | --- | --- | | `formc.get_offering()` | `Offering` | Get complete offering lifecycle | | `formc.to_context(detail)` | `str` | Token-efficient text for AI | | `formc.issuer.get_offerings()` | `List[Offering]` | All offerings by this company | | `formc.issuer.as_company()` | `Company` | Convert to full Company object | * * * Things to Know -------------- **Monetary amounts are floats.** All dollar amounts are parsed as floats and can be used directly in calculations. **Check for None before accessing optional sections.** The `offering_information` and `annual_report_disclosure` fields may be `None` depending on the form type. **Deadlines can be in the past.** Use `days_to_deadline` to check if an offering is still active. Negative values mean the deadline has passed. **Pre-revenue companies are common.** Many crowdfunding issuers have zero revenue. Use `is_pre_revenue` to detect this. **Portal file numbers identify the portal, not the offering.** To track a specific offering, use the issuer file number from `filing.file_number` or the `Offering` object. **Form C-AR has no offering information.** Annual reports focus on financial disclosures and don't include the original offering terms. **State jurisdictions are abbreviated.** The `jurisdiction` field uses state codes like "DE", "CA", "NY". * * * Related ------- * **User Guide:** Working with Form C Filings (coming soon) * **API Reference:** `edgar.offerings.formc` module * **Related Filings:** [Form D: Private Placements](https://edgartools.readthedocs.io/en/stable/guides/formd-data-object-guide/) * **Company Data:** [Find a Company](https://edgartools.readthedocs.io/en/stable/guides/finding-companies/) Back to top --- # Municipal Advisors (MA-I) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/mai-data-object-guide/#municipal-advisor-form-ma-i-data-object-guide) Municipal Advisor Form (MA-I) Data Object Guide =============================================== Overview -------- **Form MA-I** is an SEC filing required for individuals who work as municipal advisors. Municipal advisors provide advice to state and local governments on bond issuances and other municipal financial products. The SEC requires registration of these individuals to protect municipalities from unqualified or unethical advisors. The `MunicipalAdvisorForm` class in edgartools parses MA-I XML filings into structured Python objects, making it easy to extract applicant information, employment history, and critically, disclosure information about any regulatory or legal issues. Access Pattern -------------- `from edgar import Filing # Get an MA-I filing filing = Filing(form="MA-I", ...) # Parse into MunicipalAdvisorForm object ma_form = filing.obj()` * * * Core Data Structure ------------------- ### MunicipalAdvisorForm (Top-Level Object) | Property | Type | Description | | --- | --- | --- | | `filing` | `Filing` | Reference to the original SEC filing | | `filer` | `Filer` | CIK and CCC of the filing entity | | `is_amendment` | `bool` | Whether this is an amendment (MA-I/A) | | `is_individual` | `bool` | Whether applicant is an individual | | `previous_accession_no` | `str` | Accession number of prior filing if amendment | | `contact` | `Contact` | Filing contact information | | `applicant` | `Applicant` | The individual applying for registration | | `internet_notification_addresses` | `List[str]` | Email addresses for notifications | | `municipal_advisor_offices` | `List[MunicipalAdvisorOffice]` | Firms where individual is employed | | `employment_history` | `EmploymentHistory` | Current and previous employment | | `disclosures` | `Disclosures` | All disclosure questions and answers | | `signature` | `Signature` | Filing signature | * * * Applicant Information --------------------- ### Applicant The individual seeking municipal advisor registration. | Property | Type | Description | UI Usage | | --- | --- | --- | --- | | `name` | `Name` | Full legal name | Primary display | | `other_names` | `List[Name]` | Aliases, maiden names, etc. | Name history | | `crd` | `str` | FINRA CRD number | BrokerCheck link | | `number_of_advisory_firms` | `int` | Count of associated MA firms | Employment scope | | `full_name` | `str` (property) | Concatenated full name | Display convenience | ### Name | Property | Type | Description | | --- | --- | --- | | `first_name` | `str` | First name | | `middle_name` | `str` | Middle name | | `last_name` | `str` | Last name | | `suffix` | `str` | Name suffix (Jr., III, etc.) | | `full_name` | `str` (property) | Complete formatted name | ### Contact Filing contact person (may differ from applicant). | Property | Type | Description | | --- | --- | --- | | `name` | `str` | Contact person name | | `phone` | `str` | Phone number | | `email` | `str` | Email address | ### Filer | Property | Type | Description | | --- | --- | --- | | `cik` | `str` | SEC Central Index Key | | `ccc` | `str` | EDGAR Filer ID confirmation code | * * * Municipal Advisor Offices ------------------------- ### MunicipalAdvisorOffice Firms where the individual works as a municipal advisor. | Property | Type | Description | UI Usage | | --- | --- | --- | --- | | `cik` | `str` | Firm's SEC CIK | Link to firm filings | | `firm_name` | `str` | Legal name of MA firm | Firm display | | `is_independent_relationship` | `bool` | Independent contractor flag | Employment type badge | | `recent_employment_commenced_date` | `str` | When employment started | Employment timeline | | `file_number` | `str` | SEC registration file number | Registration link | | `offices` | `List[Office]` | Physical office locations | Location display | ### Office Physical office locations where the individual works. | Property | Type | Description | | --- | --- | --- | | `start_date` | `str` | When work at this location began | | `location_info` | `str` | Additional location details | | `address` | `Address` | Full address | | `street1` | `str` (property) | Street address line 1 | | `street2` | `str` (property) | Street address line 2 | | `city` | `str` (property) | City | | `state_or_country` | `str` (property) | State/country code | | `zipcode` | `str` (property) | Postal code | * * * Employment History ------------------ ### EmploymentHistory Complete work history relevant to municipal advisory activities. | Property | Type | Description | | --- | --- | --- | | `current_employer` | `Employer` | Current employment | | `previous_employers` | `List[Employer]` | Past 10 years of employment | ### Employer | Property | Type | Description | UI Usage | | --- | --- | --- | --- | | `name` | `str` | Employer name | Company display | | `start_date` | `str` | Employment start (formatted as "Jun 2015") | Timeline | | `end_date` | `str` | Employment end (None if current) | Timeline | | `ma_related` | `bool` | Municipal advisor related work | MA badge | | `investment_related` | `bool` | Investment related work | Investment badge | | `position` | `str` | Job title/position | Role display | | `address` | `Address` | Employer location | Location context | * * * Disclosures (Critical Section) ------------------------------ The disclosures section is the most important part of MA-I filings for due diligence. It contains yes/no answers to extensive questions about the applicant's regulatory, legal, and financial history. ### Disclosures (Container) | Property | Type | Description | | --- | --- | --- | | `criminal_disclosure` | `CriminalDisclosure` | Criminal history questions | | `regulatory_disclosure` | `RegulatoryDisclosure` | SEC/CFTC regulatory history | | `civil_disclosure` | `CivilDisclosure` | Civil court proceedings | | `complaint_disclosure` | `ComplaintDisclosure` | Customer complaints | | `termination_disclosure` | `TerminationDisclosure` | Employment terminations | | `financial_disclosure` | `FinancialDisclosure` | Bankruptcy/financial issues | | `judgement_lien_disclosure` | `JudgementLienDisclosure` | Judgments and liens | | `investigation_disclosure` | `InvestigationDisclosure` | Ongoing investigations | | `any()` | method | Returns `True` if ANY disclosure is positive | * * * ### CriminalDisclosure Criminal history questions. | Property | Type | Description | Red Flag Level | | --- | --- | --- | --- | | `is_convicted_of_felony` | `bool` | Felony conviction | Critical | | `is_charged_with_felony` | `bool` | Pending felony charges | Critical | | `is_org_convicted_of_felony` | `bool` | Caused org felony conviction | High | | `is_org_charged_with_felony` | `bool` | Caused org felony charges | High | | `is_convicted_of_misdemeanor` | `bool` | Misdemeanor conviction (investment-related) | Medium | | `is_charged_with_misdemeanor` | `bool` | Pending misdemeanor charges | Medium | | `is_org_convicted_of_misdemeanor` | `bool` | Caused org misdemeanor conviction | Medium | | `is_org_charged_with_misdemeanor` | `bool` | Caused org misdemeanor charges | Medium | | `any()` | method | Returns `True` if any criminal disclosure | \- | * * * ### RegulatoryDisclosure SEC, CFTC, and other regulatory agency actions. | Property | Type | Description | Red Flag Level | | --- | --- | --- | --- | | `is_made_false_statement` | `bool` | Made false statement to regulator | Critical | | `is_violated_regulation` | `bool` | Violated SEC/CFTC regulation | High | | `is_cause_of_denial` | `bool` | Caused denial of registration | High | | `is_order_against` | `bool` | Order entered against individual | High | | `is_imposed_penalty` | `bool` | Penalty imposed | High | | `is_un_ethical` | `bool` | Found dishonest or unethical | Critical | | `is_found_in_violation_of_regulation` | `bool` | Found in violation | High | | `is_found_in_cause_of_denial` | `bool` | Found to cause denial | High | | `is_order_against_activity` | `bool` | Order against activities | High | | `is_denied_license` | `bool` | License denied/suspended/revoked | Critical | | `is_found_made_false_statement` | `bool` | Found to have made false statement | Critical | | `is_found_in_violation_of_rules` | `bool` | Found in violation of rules | High | | `is_found_in_cause_of_suspension` | `bool` | Caused suspension | High | | `is_discipliend` | `bool` | Disciplined (expelled/barred) | Critical | | `is_authorized_to_act_attorney` | `bool` | Attorney authorization suspended | Medium | | `is_regulatory_complaint` | `bool` | Regulatory complaint pending | Medium | | `is_violated_security_act` | `bool` | Violated Securities Act | Critical | | `is_will_fully_aided` | `bool` | Willfully aided violation | Critical | | `is_failed_to_supervise` | `bool` | Failed to supervise | High | | `is_found_will_fully_aided` | `bool` | Found to willfully aid violation | Critical | | `is_association_bared` | `bool` | Barred from association | Critical | | `is_final_order` | `bool` | Final order entered against | High | | `is_will_fully_violated_security_act` | `bool` | Willfully violated Securities Act | Critical | | `is_failed_resonably` | `bool` | Failed reasonably to supervise | High | | `any()` | method | Returns `True` if any regulatory disclosure | \- | * * * ### CivilDisclosure Civil court proceedings. | Property | Type | Description | Red Flag Level | | --- | --- | --- | --- | | `is_enjoined` | `bool` | Enjoined in connection with MA business | High | | `is_found_violation_of_regulation` | `bool` | Court found violation | High | | `is_dismissed` | `bool` | Civil action dismissed with settlement | Medium | | `is_named_in_civil_proceeding` | `bool` | Currently named in civil proceeding | Medium | | `any()` | method | Returns `True` if any civil disclosure | \- | * * * ### ComplaintDisclosure Customer and regulatory complaints. | Property | Type | Description | Red Flag Level | | --- | --- | --- | --- | | `is_complaint_pending` | `bool` | MA-related complaint pending | Medium | | `is_complaint_settled` | `bool` | MA-related complaint settled | Low | | `is_fraud_case_pending` | `bool` | Fraud case pending | High | | `is_fraud_case_resulting_award` | `bool` | Fraud case resulted in award | High | | `is_fraud_case_settled` | `bool` | Fraud case settled | Medium | | `any()` | method | Returns `True` if any complaint disclosure | \- | * * * ### TerminationDisclosure Employment terminations under adverse circumstances. | Property | Type | Description | Red Flag Level | | --- | --- | --- | --- | | `is_violated_industry_standards` | `bool` | Terminated for violating standards | High | | `is_involved_in_fraud` | `bool` | Terminated for fraud involvement | Critical | | `is_failed_to_supervise` | `bool` | Terminated for supervision failure | High | | `any()` | method | Returns `True` if any termination disclosure | \- | * * * ### FinancialDisclosure Financial problems within past 10 years. | Property | Type | Description | Red Flag Level | | --- | --- | --- | --- | | `is_compromised` | `bool` | Made compromise with creditors | Medium | | `is_bankruptcy_petition` | `bool` | Organization filed bankruptcy | Medium | | `is_trustee_appointed` | `bool` | Trustee appointed for organization | Medium | | `is_bond_revoked` | `bool` | Bonding company denied/revoked bond | High | | `any()` | method | Returns `True` if any financial disclosure | \- | * * * ### JudgementLienDisclosure | Property | Type | Description | Red Flag Level | | --- | --- | --- | --- | | `is_lien_against` | `bool` | Currently has judgment liens | Medium | | `any()` | method | Returns `True` if any lien disclosure | \- | * * * ### InvestigationDisclosure | Property | Type | Description | Red Flag Level | | --- | --- | --- | --- | | `is_investigated` | `bool` | Currently under investigation | High | * * * Signature --------- ### Signature | Property | Type | Description | | --- | --- | --- | | `signature` | `str` | Signature text | | `date_signed` | `str` | Date of signature | | `title` | `str` | Signer's title | * * * UI Component Recommendations ---------------------------- ### Applicant Summary Card * Full name prominently displayed * CRD number (link to FINRA BrokerCheck) * Number of advisory firms * Amendment status badge * Other names (expandable) ### Disclosure Summary Panel (Critical) **This is the most important UI element for due diligence.** Display a traffic-light style indicator: - **Green**: No disclosures (`disclosures.any() == False`) - **Red**: Has disclosures (`disclosures.any() == True`) If disclosures exist, show breakdown by category: - Criminal (red if any) - Regulatory (red if any) - Civil (yellow if any) - Complaints (yellow if any) - Terminations (orange if any) - Financial (yellow if any) - Liens (yellow if any) - Under Investigation (red if true) ### Employment History Timeline * Visual timeline showing employment progression * Badges for "MA Related" and "Investment Related" work * Current employer highlighted * Employer names and positions ### Municipal Advisor Firm Details * Firm name and CIK * SEC file number (link to SEC) * Independent contractor badge * Office locations (expandable) ### Disclosure Detail Panels For each disclosure category, show individual questions with Yes/No answers: - Group by severity (Critical, High, Medium, Low) - Highlight any "Yes" answers prominently - Provide context/explanation for each question * * * Common Queries and Filters -------------------------- For SAAS features, consider enabling filters by: 1. **Clean Record** - Filter where `disclosures.any() == False` 2. **Has Disclosures** - Filter where `disclosures.any() == True` 3. **Criminal Issues** - Filter by `criminal_disclosure.any()` 4. **Regulatory Issues** - Filter by `regulatory_disclosure.any()` 5. **Under Investigation** - Filter by `investigation_disclosure.is_investigated` 6. **MA Firm** - Filter by `municipal_advisor_offices[].firm_name` 7. **State** - Filter by office state 8. **Amendment vs New** - Filter by `is_amendment` * * * Due Diligence Use Cases ----------------------- ### Red Flag Detection `# Check for any disclosures if ma_form.disclosures.any(): print("WARNING: Applicant has disclosures") # Check specific high-severity items if ma_form.disclosures.criminal_disclosure.is_convicted_of_felony: print("CRITICAL: Felony conviction") if ma_form.disclosures.regulatory_disclosure.is_association_bared: print("CRITICAL: Barred from association")` ### Employment Verification `# Get current employer current = ma_form.employment_history.current_employer print(f"Currently at: {current.name} as {current.position}") # Check MA experience for employer in ma_form.employment_history.previous_employers: if employer.ma_related: print(f"MA experience at: {employer.name}")` ### Firm Association `# List all MA firms for office in ma_form.municipal_advisor_offices: print(f"Firm: {office.firm_name}") print(f" CIK: {office.cik}") print(f" File #: {office.file_number}") print(f" Independent: {office.is_independent_relationship}")` * * * Data Quality Notes ------------------ 1. **Disclosure answers** - All stored as booleans; `True` indicates a positive disclosure 2. **Employment dates** - Formatted as "Mon YYYY" (e.g., "Jun 2015") 3. **CRD numbers** - May be empty for new registrants 4. **Other names** - May include maiden names, aliases, prior legal names 5. **Multiple offices** - An individual may work at multiple MA firm offices * * * Example Data Access ------------------- `# Get filing and parse ma_form = filing.obj() # Access applicant info print(ma_form.applicant.full_name) print(f"CRD: {ma_form.applicant.crd}") # Check for disclosures (most important!) if ma_form.disclosures.any(): print("Has disclosures - review required") # Check specific categories if ma_form.disclosures.criminal_disclosure.any(): print(" - Criminal disclosures present") if ma_form.disclosures.regulatory_disclosure.any(): print(" - Regulatory disclosures present") else: print("Clean record - no disclosures") # Employment history print(f"Current employer: {ma_form.employment_history.current_employer.name}") for emp in ma_form.employment_history.previous_employers: print(f" Previous: {emp.name} ({emp.start_date} - {emp.end_date})") # MA firm associations for office in ma_form.municipal_advisor_offices: print(f"Associated with: {office.firm_name}") # Signature verification print(f"Signed by: {ma_form.signature.signature} on {ma_form.signature.date_signed}")` * * * Form Variants ------------- The `MunicipalAdvisorForm` class handles these form types: | Form | Description | | --- | --- | | `MA-I` | Individual municipal advisor initial registration | | `MA-I/A` | Amendment to individual registration | | `MA` | Firm municipal advisor registration | | `MA/A` | Amendment to firm registration | Check `filing.form` or `is_amendment` to determine the filing type. Back to top --- # Stock Splits & EPS Normalization - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/stock-splits-eps-normalization/#stock-splits-detect-split-events-and-normalize-per-share-metrics-with-python) Stock Splits: Detect Split Events and Normalize Per-Share Metrics with Python ============================================================================= Stock splits change share counts and per-share values, making historical comparisons difficult. A company reporting $10 EPS before a 2-for-1 split should show $5 adjusted EPS for that period -- but SEC filings contain both adjusted and unadjusted values depending on when they were filed. EdgarTools detects splits from XBRL data and automatically normalizes per-share metrics so you can build consistent time series. `from edgar import Company from edgar.ttm import detect_splits company = Company("NVDA") facts = company.get_facts() splits = detect_splits(facts.get_all_facts()) for split in splits: print(f"{split['date']}: {split['ratio']:.0f}-for-1 split")` A few lines to find every stock split in a company's SEC filing history. * * * Detect Stock Splits from XBRL Data ---------------------------------- EdgarTools finds splits by looking for `StockSplitConversionRatio` facts in XBRL filings. These facts appear in 10-Q, 10-K, and 8-K reports whenever a company reports a split event. `from edgar import Company from edgar.ttm import detect_splits nvidia = Company("NVDA") entity_facts = nvidia.get_facts() facts = entity_facts.get_all_facts() splits = detect_splits(facts) for split in splits: print(f"Date: {split['date']}, Ratio: {split['ratio']}")` Output: `Date: 2021-07-20, Ratio: 4.0 Date: 2024-06-10, Ratio: 10.0` ### How Split Detection Works The `detect_splits()` function: 1. **Finds split facts** - Searches for `StockSplitConversionRatio` in XBRL concepts 2. **Filters stale data** - Rejects facts filed >280 days after the split date (historical echoes) 3. **Validates duration** - Accepts instant facts or short-duration facts (≤31 days), rejects quarterly/annual aggregations 4. **Deduplicates** - One split per year/ratio combination (same split reported in multiple filings) The ratio represents the multiplier applied to share counts. A 10-for-1 split has `ratio=10.0`. * * * Find Stock Split Announcements in 8-K Filings --------------------------------------------- Companies typically announce splits via 8-K current reports, usually under Item 8.01 ("Other Events") or occasionally Item 5.03 ("Amendments to Articles of Incorporation"). `from edgar import Company apple = Company("AAPL") filings_8k = apple.get_filings(form="8-K") # Filter to filings that might contain split announcements for filing in filings_8k[:50]: # Check recent 50 filings eight_k = filing.obj() # Look for Item 8.01 (where most splits are announced) if 'Item 8.01' in eight_k.items or 'Item 5.03' in eight_k.items: content = eight_k.get('8.01') or eight_k.get('5.03') or '' if 'split' in content.lower(): print(f"{filing.filing_date}: {filing.accession_no}") print(f"Items: {', '.join(eight_k.items)}")` Press releases attached as EX-99 exhibits often contain the split announcement details: `if eight_k.has_press_release: for release in eight_k.press_releases: text = release.text() if 'stock split' in text.lower(): print(f"Split announced: {filing.filing_date}") print(text[:500]) # First 500 chars` * * * Normalize Per-Share Metrics --------------------------- Once you've detected splits, use `apply_split_adjustments()` to retroactively adjust historical data. This makes pre-split and post-split values directly comparable. `from edgar import Company from edgar.ttm import detect_splits, apply_split_adjustments tesla = Company("TSLA") facts = tesla.get_facts().get_all_facts() # Detect splits splits = detect_splits(facts) # Apply adjustments adjusted_facts = apply_split_adjustments(facts, splits) # Compare original vs adjusted EPS eps_facts = [f for f in facts if 'EarningsPerShare' in f.concept] eps_adjusted = [f for f in adjusted_facts if 'EarningsPerShare' in f.concept] for orig, adj in zip(eps_facts[:3], eps_adjusted[:3]): print(f"{orig.period_end}: ${orig.numeric_value:.2f} → ${adj.numeric_value:.2f}")` Output shows EPS before and after split adjustment: `2021-12-31: $4.90 → $1.63 2022-03-31: $3.22 → $1.07 2022-06-30: $2.27 → $0.76` ### What Gets Adjusted The adjustment logic depends on the fact's unit and concept: | Type | Unit Pattern | Adjustment | Examples | | --- | --- | --- | --- | | **Per-share metrics** | `/share` in unit or `EarningsPerShare` in concept | Divide by ratio | EPS, Dividends per share, Book value per share | | **Share counts** | `shares` in unit (but not per-share) | Multiply by ratio | Shares outstanding, Weighted average shares | | **Other metrics** | All others | No adjustment | Revenue, Net Income, Assets | `# Per-share: Divide by split ratio # If 10-for-1 split, $10 EPS becomes $1 EPS adjusted_eps = original_eps / 10.0 # Share counts: Multiply by split ratio # If 10-for-1 split, 100M shares becomes 1B shares adjusted_shares = original_shares * 10.0` ### Retroactive Adjustment Rules Splits only adjust facts from periods **before** the split date. The function applies cumulative ratios for multiple splits: `# Example: Company had two splits # 2021-07-20: 4-for-1 split # 2024-06-10: 10-for-1 split # For a fact from 2020: # - Both splits occurred after 2020 # - Cumulative ratio = 4.0 * 10.0 = 40.0 # - Adjust by dividing by 40 # For a fact from 2022: # - Only 2024 split occurred after 2022 # - Cumulative ratio = 10.0 # - Adjust by dividing by 10 # For a fact from 2024-07: # - No splits after this date # - No adjustment needed (already post-split values)` The function also checks filing dates. If a fact was filed **after** the split, it's already adjusted by the company and doesn't need further modification. * * * Automatic Split Handling in TTM Calculations -------------------------------------------- TTM (Trailing Twelve Months) methods automatically detect and apply split adjustments. You don't need to call `detect_splits()` or `apply_split_adjustments()` manually when using these methods. `from edgar import Company nvidia = Company("NVDA") # TTM calculations handle splits automatically ttm_revenue = nvidia.get_ttm_revenue() ttm_net_income = nvidia.get_ttm_net_income() print(f"TTM Revenue: ${ttm_revenue.value / 1e9:.1f}B") print(f"TTM Net Income: ${ttm_net_income.value / 1e9:.1f}B") print(f"As of: {ttm_revenue.as_of_date}") print(f"Periods: {ttm_revenue.periods}")` For any XBRL concept: `# Per-share metrics are automatically split-adjusted ttm_eps = nvidia.get_ttm("EarningsPerShareBasic") print(f"TTM EPS: ${ttm_eps.value:.2f}") # Share counts are automatically adjusted too ttm_shares = nvidia.get_ttm("WeightedAverageNumberOfSharesOutstandingBasic") print(f"Weighted Avg Shares: {ttm_shares.value / 1e9:.2f}B")` Behind the scenes, `get_ttm()` calls the internal method `_get_split_adjusted_facts()` which: 1. Gets all facts for the company 2. Detects splits using `detect_splits()` 3. Applies adjustments using `apply_split_adjustments()` 4. Returns normalized facts for TTM calculation * * * Complete Workflow: NVIDIA 10-for-1 Split Example ------------------------------------------------ NVIDIA executed a 10-for-1 stock split on June 10, 2024. Let's build a complete workflow that detects this split, normalizes historical EPS, and validates the adjustments. `from edgar import Company from edgar.ttm import detect_splits, apply_split_adjustments # Step 1: Get NVIDIA data nvidia = Company("NVDA") facts = nvidia.get_facts().get_all_facts() # Step 2: Detect splits splits = detect_splits(facts) print(f"Found {len(splits)} splits:") for split in splits: print(f" {split['date']}: {split['ratio']:.0f}-for-1") # Step 3: Filter to EPS facts eps_facts = [f for f in facts if 'EarningsPerShareBasic' in f.concept and f.period_end and f.numeric_value is not None] # Sort by period eps_facts.sort(key=lambda f: f.period_end) # Step 4: Apply split adjustments adjusted_facts = apply_split_adjustments(eps_facts, splits) # Step 5: Compare pre-split periods print("\nEPS Comparison (split-adjusted):") print(f"{'Period':<12} {'Original':>12} {'Adjusted':>12} {'Cumulative Ratio':>18}") print("-" * 58) for orig, adj in zip(eps_facts[-8:], adjusted_facts[-8:]): # Calculate cumulative ratio from the adjustment context context = adj.calculation_context or "" if "split_adj_ratio" in context: ratio = float(context.split("_")[-1]) else: ratio = 1.0 print(f"{orig.period_end!s:<12} ${orig.numeric_value:>11.2f} " f"${adj.numeric_value:>11.2f} {ratio:>17.1f}x")` Expected output: `Found 2 splits: 2021-07-20: 4-for-1 2024-06-10: 10-for-1 EPS Comparison (split-adjusted): Period Original Adjusted Cumulative Ratio ---------------------------------------------------------- 2022-01-30 $ 4.44 $ 0.11 40.0x 2022-05-01 $ 1.36 $ 0.03 40.0x 2022-07-31 $ 0.51 $ 0.01 40.0x 2022-10-30 $ 0.58 $ 0.01 40.0x 2023-01-29 $ 0.88 $ 0.02 40.0x 2023-04-30 $ 1.09 $ 0.03 40.0x 2023-07-30 $ 2.70 $ 0.07 40.0x 2024-01-28 $ 5.16 $ 0.13 40.0x` Periods before both splits get cumulative adjustment of 40x (4 × 10). Periods between the splits would get 10x adjustment. Periods after June 2024 need no adjustment. * * * Common Analysis Patterns ------------------------ ### Track Split History Across Multiple Companies `from edgar import Company from edgar.ttm import detect_splits tech_stocks = ["AAPL", "MSFT", "GOOGL", "AMZN", "NVDA", "TSLA"] for ticker in tech_stocks: try: company = Company(ticker) facts = company.get_facts() splits = detect_splits(facts.get_all_facts()) if splits: print(f"\n{ticker} ({company.name}):") for split in splits: print(f" {split['date']}: {split['ratio']:.1f}-for-1") else: print(f"\n{ticker}: No splits detected") except Exception as e: print(f"\n{ticker}: Error - {e}")` ### Build Split-Adjusted Time Series Compare EPS over multiple years with consistent per-share values: `from edgar import Company from edgar.ttm import detect_splits, apply_split_adjustments import pandas as pd company = Company("AAPL") facts = company.get_facts().get_all_facts() # Get splits and adjust facts splits = detect_splits(facts) adjusted_facts = apply_split_adjustments(facts, splits) # Extract EPS time series eps_data = [] for f in adjusted_facts: if 'EarningsPerShareBasic' in f.concept and f.fiscal_period == 'FY': eps_data.append({ 'fiscal_year': f.fiscal_year, 'period_end': f.period_end, 'eps': f.numeric_value }) # Create DataFrame df = pd.DataFrame(eps_data).sort_values('fiscal_year') print(df) # Calculate growth rates on split-adjusted data df['yoy_growth'] = df['eps'].pct_change() * 100 print(f"\nAverage EPS growth: {df['yoy_growth'].mean():.1f}%")` ### Validate Split Adjustments Against Company Reports Companies publish adjusted historical data after splits. You can validate the adjustment logic: `from edgar import Company from edgar.ttm import detect_splits, apply_split_adjustments company = Company("NVDA") facts = company.get_facts().get_all_facts() splits = detect_splits(facts) adjusted = apply_split_adjustments(facts, splits) # Find the same period reported before and after split # Pre-split: Filed in 2023 10-K, reports 2023 EPS (pre-split basis) # Post-split: Filed in 2024 10-K, reports 2023 EPS (post-split basis) eps_2023_filings = [f for f in adjusted if 'EarningsPerShareBasic' in f.concept and f.fiscal_year == 2023] # Group by filing date from collections import defaultdict by_filing = defaultdict(list) for f in eps_2023_filings: by_filing[f.filing_date].append(f) # Compare values across filings for filing_date in sorted(by_filing.keys())[:3]: facts_list = by_filing[filing_date] if facts_list: f = facts_list[0] print(f"Filed {filing_date}: 2023 EPS = ${f.numeric_value:.2f}")` ### Calculate Split-Adjusted Market Cap History Combine share counts and stock prices to build historical market cap: `from edgar import Company from edgar.ttm import detect_splits, apply_split_adjustments company = Company("AAPL") facts = company.get_facts().get_all_facts() splits = detect_splits(facts) adjusted = apply_split_adjustments(facts, splits) # Get split-adjusted shares outstanding shares_facts = [f for f in adjusted if 'CommonStockSharesOutstanding' in f.concept and f.fiscal_period == 'FY'] for f in sorted(shares_facts, key=lambda x: x.period_end)[-5:]: shares_b = f.numeric_value / 1e9 print(f"{f.period_end}: {shares_b:.2f}B shares (split-adjusted)")` * * * API Quick Reference ------------------- ### Detection and Adjustment Functions | Function | Parameters | Returns | Description | | --- | --- | --- | --- | | `detect_splits(facts)` | `facts`: List of FinancialFact | List of dicts with `date` and `ratio` | Find all stock splits in facts | | `apply_split_adjustments(facts, splits)` | `facts`: List of FinancialFact
`splits`: List of split dicts | List of adjusted FinancialFact | Apply retroactive adjustments | ### Company Methods with Automatic Split Handling | Method | Returns | Description | | --- | --- | --- | | `company.get_ttm(concept, as_of=None)` | `TTMMetric` | TTM for any concept (split-adjusted) | | `company.get_ttm_revenue(as_of=None)` | `TTMMetric` | TTM revenue (split-adjusted) | | `company.get_ttm_net_income(as_of=None)` | `TTMMetric` | TTM net income (split-adjusted) | ### Split Detection Parameters The detection logic uses these constants from `edgar.ttm.calculator`: | Constant | Value | Purpose | | --- | --- | --- | | `MAX_SPLIT_LAG_DAYS` | 280 | Maximum days between split date and filing date | | `MAX_SPLIT_DURATION_DAYS` | 31 | Maximum period duration for split facts | * * * Things to Know -------------- **Splits apply retroactively to historical data only.** Facts from periods after the split date don't need adjustment -- they're already reported on a post-split basis. **Filing date matters for restated facts.** If a fact was filed after a split date, the company has already adjusted it. The adjustment logic checks `filing_date` to avoid double-adjusting. **Multiple splits compound.** A company with a 4-for-1 split in 2021 and a 10-for-1 split in 2024 requires a cumulative adjustment of 40x for pre-2021 data. **Reverse splits work automatically.** A 1-for-10 reverse split has `ratio=0.1`. Per-share metrics get divided by 0.1 (multiplied by 10), which correctly increases the adjusted historical EPS. **Balance sheet items don't need adjustment.** Assets, liabilities, and equity are not per-share values. Total stockholders' equity stays the same regardless of share count. **Not all facts have filing dates.** The adjustment logic handles `None` filing dates by assuming the fact needs adjustment if it's from before the split. **Instant facts are preferred.** Split events are moment-in-time occurrences. The detector accepts instant facts (no `period_start`) or short-duration facts (≤31 days) but rejects quarterly/annual durations. **8-K filing timing varies.** While Item 8.01 is common for split announcements, check Items 5.03 and exhibit press releases. Not all companies follow the same disclosure pattern. **Weighted average share counts need special handling.** These represent time-weighted averages over a period. For Q4 EPS derivation with splits, use the formula: Q4 shares = 4 × annual\_shares - 3 × YTD\_9M\_shares. * * * Related ------- * [Financial Data](https://edgartools.readthedocs.io/en/latest/guides/financial-data/) - Extract financial statements and metrics * [8-K Current Reports](https://edgartools.readthedocs.io/en/latest/eightk-filings/) - Parse material event filings * [Company Facts API](https://edgartools.readthedocs.io/en/latest/guides/company-facts/) - Access XBRL facts programmatically Back to top --- # Dimensions - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/xbrl/concepts/dimension-handling/#understanding-dimensions-in-financial-statements) Understanding Dimensions in Financial Statements ================================================ This guide explains how EdgarTools handles XBRL dimensions and how to get complete, accurate financial statements. Quick Summary ------------- When you retrieve a financial statement, EdgarTools automatically: - **Shows** values that belong on the face of the statement (including dimensional face values) - **Hides** breakdown details that belong in notes disclosures (geographic, segment, etc.) `from edgar import Company company = Company("WDAY") filing = company.get_filings(form="10-K").latest() xbrl = filing.xbrl() # Default: Shows face presentation (what you'd see in SEC Viewer) income = xbrl.statements.income_statement() print(income) # Full data: Shows everything for custom analysis df = income.to_dataframe(view="detailed")` Why Dimensions Matter --------------------- Many companies report financial values **only through dimensional XBRL**. Without proper handling, these statements appear incomplete or out-of-balance. ### The Scale of the Problem Based on community research ([GH-577](https://github.com/dgunning/edgartools/issues/577) ), dimensional-only reporting is widespread: **Income Statement - Cost of Goods Sold:** - BA (Boeing) 2023+, CARR (Carrier) 2020+, GD (General Dynamics) 2020+ - HII (Huntington Ingalls) 2020+, INTU (Intuit) 2018+, NOC (Northrop Grumman) 2019+ - RTX 2019+, SLB (Schlumberger) 2018+, WDAY (Workday) 2019+ - CHH 2022+, CHRW 2018+, CTAS 2020+, GEHC 2024+, MAR 2022+ - OTIS 2020+, PFE 2022, TT 2021+, UPS 2018, VZ 2020+ **Balance Sheet - Various Line Items:** - Goodwill: BSX 2019, IBM 2023+, JKHY 2016+, MCD 2023 - PPE: BSX 2019, CSX 2015+, HLT 2022+, PFE 2022 - Receivables: COP 2023+, FIS 2024+, GEHC 2023+, LYB 2023+ - Payables: COP 2023+, HLT 2019+, LYB 2023+, WDC 2023+ - Debt: ADP 2023+, CAT 2020+, HLT 2019+ - Contract Liabilities: BBY 2021+, HLT 2019+, MAR 2018+, REGN 2020 ### Example: Workday Income Statement Workday reports Cost of Goods Sold exclusively via `ProductOrServiceAxis`: `2025 2024 2023 Revenue: Subscription services $7,718M $6,603M $5,567M Professional services $728M $656M $649M Total Revenue $8,446M $7,259M $6,216M Cost of Goods Sold: Subscription services $1,266M $1,031M $1,007M Professional services $803M $740M $703M Total COGS $2,069M $1,771M $1,710M` **Without dimensional handling**: COGS would show as NaN/missing, making it impossible to calculate gross margin. **With EdgarTools**: Both subscription and professional services COGS values are preserved, and the statement balances correctly. ### Example: Caterpillar Balance Sheet Caterpillar reports debt through dimensional XBRL across multiple years: | Year | Concept | Axis Used | | --- | --- | --- | | 2020-2025 | ShortTermBorrowings | ConsolidationItemsAxis | | 2020-2025 | LongTermDebtCurrent | ConsolidationItemsAxis | | 2020-2025 | LongTermDebt | ConsolidationItemsAxis | EdgarTools preserves these values so debt totals appear correctly on the balance sheet. How It Works ------------ ### Face Values vs Breakdowns Not all dimensional data is the same: | Type | Description | Example | Shown by Default? | | --- | --- | --- | --- | | **Face Value** | Values that appear on the statement face | Product vs Service revenue | ✅ Yes | | **Breakdown** | Drill-down details for notes disclosures | Revenue by country | ❌ No | > **Member Hierarchy (v5.21.1+):** When using `view="detailed"`, sub-members within a dimension are now properly nested under their parent members. For example, Tesla's "Automotive sales" and "Automotive regulatory credits" appear as children of "Automotive Revenues" with increasing `level` values, reflecting the definition linkbase hierarchy. ### Classification Logic EdgarTools uses a tiered approach to classify dimensions: **Tier 1: Definition Linkbase (Authoritative)** - The XBRL filing itself declares which dimensions are valid for each statement - If declared in the definition linkbase, it's a face value - Highest confidence classification **Tier 2: Curated Axis Lists** - Known face-level axes: `ProductOrServiceAxis`, `DebtInstrumentAxis`, `PropertyPlantAndEquipmentByTypeAxis` - Known breakdown axes: `StatementGeographicalAxis`, `StatementBusinessSegmentsAxis`, `BusinessAcquisitionAxis` - Based on empirical analysis of S&P 500 filings **Tier 3: Pattern Matching** - Axes matching patterns like `FairValue*Axis` or `*HierarchyLevelAxis` are classified as breakdowns - Fallback when other methods don't apply Usage Guide ----------- ### Standard View (Default) Get the statement as it would appear in the SEC Viewer: `from edgar import Company company = Company("SLB") filing = company.get_filings(form="10-K").latest() xbrl = filing.xbrl() # Face presentation - includes dimensional face values income = xbrl.statements.income_statement() print(income)` Output shows COGS by Product and Services (the dimensional face values): `Cost of Goods and Services Sold: Product $10,982M Services $17,847M` ### Full Data View Get all data including breakdowns for custom analysis: `# All dimensional data included — use view="detailed" df = income.to_dataframe(view="detailed") # Filter as needed for your analysis geographic_breakdown = df[df['dimension_label'].str.contains('Geographic', na=False)]` ### Working with Dimensional Data The dataframe includes helpful columns for understanding dimensions: `df = income.to_dataframe(view="detailed") # Key columns: # - 'dimension': True/False - is this a dimensional row? # - 'is_breakdown': True/False - is this a breakdown (vs face value)? # - 'dimension_label': Human-readable dimension info # Find all face-level dimensional values face_dimensional = df[(df['dimension'] == True) & (df['is_breakdown'] == False)] # Find all breakdown values breakdowns = df[df['is_breakdown'] == True]` ### Calculating Totals When a concept has dimensional values but no non-dimensional total, you may need to sum: `df = income.to_dataframe(view="standard") # COGS may have individual values but NaN for total cogs_rows = df[df['concept'] == 'us-gaap_CostOfGoodsAndServicesSold'] # Sum the non-NaN values for the total period_col = '2025-01-31' # or whichever period you need total_cogs = cogs_rows[period_col].sum()` Dimension Classification Reference ---------------------------------- ### Face-Level Axes (Always Shown) These dimensions represent valid face presentation and are preserved by default: | Axis | Usage | | --- | --- | | `ProductOrServiceAxis` | Product vs Service breakdown (revenue, COGS) | | `PropertyPlantAndEquipmentByTypeAxis` | PPE categories | | `DebtInstrumentAxis` | Debt instrument types | | `LongtermDebtTypeAxis` | Long-term debt categories | | `ShortTermDebtTypeAxis` | Short-term debt categories | | `StatementClassOfStockAxis` | Stock class distinctions | | `ContracttypeAxis` | Contract types (defense contractors) | | `MajorProgramsAxis` | Major program breakdown (defense) | ### Breakdown Axes (Filtered by Default) These dimensions represent notes disclosures and are hidden by default: | Axis | Usage | | --- | --- | | `StatementGeographicalAxis` | Geographic segment breakdown | | `StatementBusinessSegmentsAxis` | Business segment breakdown | | `BusinessAcquisitionAxis` | Acquisition-specific details | | `ConsolidationItemsAxis` | Consolidation eliminations | | `MajorCustomersAxis` | Customer concentration | | `RestatementAxis` | Prior period adjustments | | `FairValueByFairValueHierarchyLevelAxis` | Fair value hierarchy | | `RetirementPlanTypeAxis` | Pension plan details | ### Context-Dependent Axes Some axes behave differently based on statement type: `# StatementEquityComponentsAxis: # - On Statement of Equity: STRUCTURAL (defines columns) - shown # - On Balance Sheet: BREAKDOWN (notes detail) - hidden` Troubleshooting --------------- ### Statement Shows NaN for Expected Values **Possible causes:** 1. **Dimensional-only value with old EdgarTools version**: Upgrade to v5.7.4+ 2. **No total row exists**: The XBRL only has dimensional breakdown, no aggregated total 3. **Unknown axis**: The dimension axis isn't in our classification lists **Solution:** `# Check what's in the full data df = statement.to_dataframe(view="detailed") concept_rows = df[df['concept'].str.contains('YourConcept')] print(concept_rows[['label', 'dimension', 'dimension_label', value_column]])` ### Statement Doesn't Balance **Check the dimensional data:** `df = income.to_dataframe(view="detailed") # Look for missing values that might be dimensional missing = df[df[value_column].isna() & (df['abstract'] == False)] print(missing[['concept', 'label', 'dimension']])` ### Need a Specific Breakdown `# Get all data first df = statement.to_dataframe(view="detailed") # Filter to specific dimension geographic = df[df['dimension_label'].str.contains('Geographic', na=False)]` API Reference ------------- ### Statement Methods `# Get statement with default handling (face values preserved) statement = xbrl.statements.income_statement() # Control dimensional data with the view parameter df = statement.to_dataframe(view="standard") # Face presentation (default for display) df = statement.to_dataframe(view="detailed") # All dimensional data included df = statement.to_dataframe(view="summary") # Non-dimensional totals only # The view parameter also works on stitched (multi-period) statements income = xbrls.statements.income_statement(view="detailed") df = income.to_dataframe()` The `view` parameter accepts `"standard"`, `"detailed"`, or `"summary"` (or the `StatementView` enum). The legacy `include_dimensions` boolean is still supported but `view` is preferred. ### Dimension Classification API `from edgar.xbrl.dimensions import ( classify_dimension_with_confidence, DimensionConfidence, is_breakdown_dimension, ) # Check if an item is a breakdown is_breakdown = is_breakdown_dimension(item, xbrl=xbrl, role_uri=role_uri) # Get detailed classification classification, confidence, reason = classify_dimension_with_confidence( item, xbrl=xbrl, role_uri=role_uri ) # Returns: ('face', DimensionConfidence.HIGH, 'Declared in definition linkbase')` ### XBRL Dimension Methods `# Check if definition linkbase exists for a role has_def = xbrl.has_definition_linkbase_for_role(role_uri) # Check if a specific dimension is valid for a role is_valid = xbrl.is_dimension_valid_for_role('srt:ProductOrServiceAxis', role_uri) # Get all valid dimensions for a role valid_dims = xbrl.get_valid_dimensions_for_role(role_uri)` Version History --------------- | Version | Change | | --- | --- | | v5.21.1 | Member hierarchy support — sub-members nested under parent members in `to_dataframe(view="detailed")` using definition linkbase hierarchy | | v5.7.4 | Definition linkbase-based dimension filtering (GH-577 fix) | | v5.7.2 | Initial dimension handling with hardcoded lists (GH-569) | | v5.7.0 | Changed default to `include_dimensions=False` | Related Resources ----------------- * [XBRL Documentation Hub](https://edgartools.readthedocs.io/en/latest/xbrl/) - Central navigation for all XBRL docs * [Extract Financial Statements Guide](https://edgartools.readthedocs.io/en/latest/guides/extract-statements/) - Complete guide to extracting financial data * [XBRL Standardization Concepts](https://edgartools.readthedocs.io/en/latest/xbrl/concepts/standardization/) - 95 standard concepts for cross-company comparison * [Multi-Period Analysis](https://edgartools.readthedocs.io/en/latest/xbrl/guides/multi-period-analysis/) - Working with multiple filings * [GitHub Issue #577](https://github.com/dgunning/edgartools/issues/577) - Original problem documentation * [SEC Financial Statement Data Sets](https://www.sec.gov/data-research/sec-markets-data/financial-statement-notes-data-sets) - SEC's processed XBRL data Acknowledgments --------------- Special thanks to [@mpreiss9](https://github.com/mpreiss9) for extensive research documenting dimensional-only reporting patterns across hundreds of filings, which directly informed this implementation. Back to top --- # Footnotes - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/xbrl-footnotes/#xbrl-footnotes) XBRL Footnotes ============== EdgarTools now supports parsing and accessing footnotes from XBRL documents. Footnotes provide additional context and explanations for specific facts in financial statements. Overview -------- In XBRL documents, footnotes are linked to facts through a structured relationship: - **Facts** have unique `id` attributes (e.g., `id="ID_123"`) - **Footnotes** contain explanatory text and have their own identifiers - **FootnoteArcs** connect facts to footnotes using XLink references EdgarTools automatically extracts these relationships, making footnotes easily accessible alongside the financial data. Basic Usage ----------- ### Accessing Footnotes `from edgar.xbrl.parser import XBRLParser # Parse an XBRL instance document parser = XBRLParser() with open("instance.xml") as f: content = f.read() parser.parse_instance_content(content) # Access footnotes print(f"Found {len(parser.footnotes)} footnotes") # Iterate through footnotes for footnote_id, footnote in parser.footnotes.items(): print(f"Footnote {footnote_id}: {footnote.text}") print(f"Related to facts: {footnote.related_fact_ids}")` ### Finding Facts with Footnotes `# Find facts that have footnotes facts_with_footnotes = [ fact for fact in parser.facts.values() if fact.footnotes ] print(f"Found {len(facts_with_footnotes)} facts with footnotes") # Show fact details for fact in facts_with_footnotes[:5]: print(f"Fact: {fact.element_id} (ID: {fact.fact_id})") print(f"Value: {fact.value}") print(f"Footnotes: {', '.join(fact.footnotes)}") print()` Using the XBRL Class -------------------- The XBRL class provides convenient methods for working with footnotes: `from edgar.xbrl import XBRL # Initialize and parse xbrl = XBRL() xbrl.parser.parse_instance_content(content) # Access footnotes property footnotes = xbrl.footnotes print(f"Document has {len(footnotes)} footnotes") # Get footnotes for a specific fact ID fact_footnotes = xbrl.get_footnotes_for_fact("ID_123") for footnote in fact_footnotes: print(f"Footnote: {footnote.text}") # Get all facts that have footnotes facts_with_footnotes = xbrl.get_facts_with_footnotes()` Data Models ----------- ### Fact Model The `Fact` model has been enhanced with footnote support: `class Fact(BaseModel): element_id: str context_ref: str value: str unit_ref: Optional[str] = None decimals: Optional[Union[int, str]] = None numeric_value: Optional[float] = None footnotes: List[str] = Field(default_factory=list) # Footnote IDs instance_id: Optional[int] = None fact_id: Optional[str] = None # Original XML id attribute` ### Footnote Model `class Footnote(BaseModel): footnote_id: str text: str lang: Optional[str] = "en-US" role: Optional[str] = None related_fact_ids: List[str] = Field(default_factory=list)` Real-World Example ------------------ Here's a complete example using a filing with footnotes: `from edgar import Filing from pathlib import Path # Get a filing and parse its XBRL filing = Filing(form='10-K', cik=1234567, accession_no='0001234567-23-000001') xbrl = filing.xbrl() # Check if the document has footnotes if xbrl.footnotes: print(f"Document contains {len(xbrl.footnotes)} footnotes") # Show footnote details for footnote_id, footnote in list(xbrl.footnotes.items())[:3]: print(f"\nFootnote ID: {footnote_id}") print(f"Text: {footnote.text[:100]}...") print(f"Language: {footnote.lang}") print(f"Linked to {len(footnote.related_fact_ids)} facts") # Find facts with footnotes in the balance sheet balance_sheet = xbrl.get_statement("BalanceSheet") if balance_sheet: for item in balance_sheet.get_all_line_items(): facts = item.get("facts", []) for fact in facts: if fact.footnotes: print(f"\n{item['label']} has footnotes:") for fn_id in fact.footnotes: if fn_id in xbrl.footnotes: print(f" • {xbrl.footnotes[fn_id].text[:80]}...") else: print("No footnotes found in this document")` Advanced Usage -------------- ### Filtering Footnotes by Content `# Find footnotes containing specific keywords debt_footnotes = [ (fn_id, footnote) for fn_id, footnote in parser.footnotes.items() if any(keyword in footnote.text.lower() for keyword in ['debt', 'loan', 'credit']) ] print(f"Found {len(debt_footnotes)} footnotes related to debt")` ### Creating a Footnote Report `from rich.console import Console from rich.table import Table console = Console() # Create a footnote summary table table = Table(title="Footnote Summary", show_header=True) table.add_column("ID", style="cyan") table.add_column("Preview", style="white", width=60) table.add_column("Facts", style="yellow", justify="right") for fn_id, footnote in parser.footnotes.items(): preview = footnote.text[:60] + "..." if len(footnote.text) > 60 else footnote.text fact_count = str(len(footnote.related_fact_ids)) table.add_row(fn_id, preview, fact_count) console.print(table)` ### Cross-Referencing with Financial Statements `# Find footnotes that reference specific financial statement items def find_footnotes_for_concept(xbrl, concept_name): """Find footnotes related to a specific accounting concept.""" related_footnotes = [] # Find facts matching the concept for fact_key, fact in xbrl.parser.facts.items(): if concept_name.lower() in fact.element_id.lower(): if fact.footnotes: for fn_id in fact.footnotes: if fn_id in xbrl.footnotes: related_footnotes.append((fact, xbrl.footnotes[fn_id])) return related_footnotes # Example: Find footnotes about revenue revenue_footnotes = find_footnotes_for_concept(xbrl, "Revenue") for fact, footnote in revenue_footnotes: print(f"Revenue fact {fact.fact_id}: {footnote.text}")` XBRL Technical Details ---------------------- ### Footnote Structure in XBRL XBRL footnotes follow the XBRL 2.1 specification: ` Explanatory text about the fact. ` ### Supported Footnote Formats EdgarTools handles: - **Standard footnotes** with `id` attributes - **XLink footnotes** with `xlink:label` attributes \- **XHTML content** within footnotes (automatically extracts text) - **Multiple languages** via `xml:lang` attributes - **Custom roles** via `xlink:role` attributes ### Namespace Handling The parser correctly handles all standard XBRL namespaces: - `http://www.xbrl.org/2003/linkbase` (link) - `http://www.w3.org/1999/xlink` (xlink) - `http://www.w3.org/1999/xhtml` (xhtml) - `http://www.w3.org/XML/1998/namespace` (xml) Performance Considerations -------------------------- * Footnote extraction adds minimal overhead to XBRL parsing * Footnotes are parsed lazily during instance document processing * Both fact-to-footnote and footnote-to-fact lookups are O(1) operations * Large documents with many footnotes are handled efficiently Error Handling -------------- The parser gracefully handles common footnote issues: `# Parser warnings for missing footnote references # Warning: "Footnote arc references undefined footnote: footnote_123" # Missing footnote definitions are logged but don't cause parsing to fail # Malformed XHTML content is handled with fallback text extraction` Migration from Manual Parsing ----------------------------- If you were previously parsing footnotes manually: `# Before (manual parsing) import xml.etree.ElementTree as ET def extract_footnotes_manually(xml_content): root = ET.fromstring(xml_content) footnotes = {} # ... complex manual parsing logic ... return footnotes # After (using EdgarTools) from edgar.xbrl.parser import XBRLParser parser = XBRLParser() parser.parse_instance_content(xml_content) footnotes = parser.footnotes # Ready to use!` API Reference ------------- ### XBRLParser.footnotes * **Type**: `Dict[str, Footnote]` * **Description**: Dictionary mapping footnote IDs to Footnote objects ### XBRL.footnotes * **Type**: `Dict[str, Footnote]` * **Description**: Property providing access to parser footnotes ### XBRL.get\_footnotes\_for\_fact(fact\_id: str) * **Parameters**: `fact_id` - The ID of the fact to get footnotes for * **Returns**: `List[Footnote]` - List of associated footnotes * **Description**: Retrieves all footnotes linked to a specific fact ### XBRL.get\_facts\_with\_footnotes() * **Returns**: `Dict[str, Fact]` - Dictionary of facts that have footnotes * **Description**: Returns all facts that reference footnotes ### Fact.footnotes * **Type**: `List[str]` * **Description**: List of footnote IDs that reference this fact ### Fact.fact\_id * **Type**: `Optional[str]` * **Description**: Original `id` attribute from the XML element * * * _This feature was implemented to support the XBRL 2.1 specification for footnotes and is compatible with all standard SEC XBRL filings._ Back to top --- # Footnotes - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/xbrl-footnotes/#xbrl-footnotes) XBRL Footnotes ============== EdgarTools now supports parsing and accessing footnotes from XBRL documents. Footnotes provide additional context and explanations for specific facts in financial statements. Overview -------- In XBRL documents, footnotes are linked to facts through a structured relationship: - **Facts** have unique `id` attributes (e.g., `id="ID_123"`) - **Footnotes** contain explanatory text and have their own identifiers - **FootnoteArcs** connect facts to footnotes using XLink references EdgarTools automatically extracts these relationships, making footnotes easily accessible alongside the financial data. Basic Usage ----------- ### Accessing Footnotes `from edgar.xbrl.parser import XBRLParser # Parse an XBRL instance document parser = XBRLParser() with open("instance.xml") as f: content = f.read() parser.parse_instance_content(content) # Access footnotes print(f"Found {len(parser.footnotes)} footnotes") # Iterate through footnotes for footnote_id, footnote in parser.footnotes.items(): print(f"Footnote {footnote_id}: {footnote.text}") print(f"Related to facts: {footnote.related_fact_ids}")` ### Finding Facts with Footnotes `# Find facts that have footnotes facts_with_footnotes = [ fact for fact in parser.facts.values() if fact.footnotes ] print(f"Found {len(facts_with_footnotes)} facts with footnotes") # Show fact details for fact in facts_with_footnotes[:5]: print(f"Fact: {fact.element_id} (ID: {fact.fact_id})") print(f"Value: {fact.value}") print(f"Footnotes: {', '.join(fact.footnotes)}") print()` Using the XBRL Class -------------------- The XBRL class provides convenient methods for working with footnotes: `from edgar.xbrl import XBRL # Initialize and parse xbrl = XBRL() xbrl.parser.parse_instance_content(content) # Access footnotes property footnotes = xbrl.footnotes print(f"Document has {len(footnotes)} footnotes") # Get footnotes for a specific fact ID fact_footnotes = xbrl.get_footnotes_for_fact("ID_123") for footnote in fact_footnotes: print(f"Footnote: {footnote.text}") # Get all facts that have footnotes facts_with_footnotes = xbrl.get_facts_with_footnotes()` Data Models ----------- ### Fact Model The `Fact` model has been enhanced with footnote support: `class Fact(BaseModel): element_id: str context_ref: str value: str unit_ref: Optional[str] = None decimals: Optional[Union[int, str]] = None numeric_value: Optional[float] = None footnotes: List[str] = Field(default_factory=list) # Footnote IDs instance_id: Optional[int] = None fact_id: Optional[str] = None # Original XML id attribute` ### Footnote Model `class Footnote(BaseModel): footnote_id: str text: str lang: Optional[str] = "en-US" role: Optional[str] = None related_fact_ids: List[str] = Field(default_factory=list)` Real-World Example ------------------ Here's a complete example using a filing with footnotes: `from edgar import Filing from pathlib import Path # Get a filing and parse its XBRL filing = Filing(form='10-K', cik=1234567, accession_no='0001234567-23-000001') xbrl = filing.xbrl() # Check if the document has footnotes if xbrl.footnotes: print(f"Document contains {len(xbrl.footnotes)} footnotes") # Show footnote details for footnote_id, footnote in list(xbrl.footnotes.items())[:3]: print(f"\nFootnote ID: {footnote_id}") print(f"Text: {footnote.text[:100]}...") print(f"Language: {footnote.lang}") print(f"Linked to {len(footnote.related_fact_ids)} facts") # Find facts with footnotes in the balance sheet balance_sheet = xbrl.get_statement("BalanceSheet") if balance_sheet: for item in balance_sheet.get_all_line_items(): facts = item.get("facts", []) for fact in facts: if fact.footnotes: print(f"\n{item['label']} has footnotes:") for fn_id in fact.footnotes: if fn_id in xbrl.footnotes: print(f" • {xbrl.footnotes[fn_id].text[:80]}...") else: print("No footnotes found in this document")` Advanced Usage -------------- ### Filtering Footnotes by Content `# Find footnotes containing specific keywords debt_footnotes = [ (fn_id, footnote) for fn_id, footnote in parser.footnotes.items() if any(keyword in footnote.text.lower() for keyword in ['debt', 'loan', 'credit']) ] print(f"Found {len(debt_footnotes)} footnotes related to debt")` ### Creating a Footnote Report `from rich.console import Console from rich.table import Table console = Console() # Create a footnote summary table table = Table(title="Footnote Summary", show_header=True) table.add_column("ID", style="cyan") table.add_column("Preview", style="white", width=60) table.add_column("Facts", style="yellow", justify="right") for fn_id, footnote in parser.footnotes.items(): preview = footnote.text[:60] + "..." if len(footnote.text) > 60 else footnote.text fact_count = str(len(footnote.related_fact_ids)) table.add_row(fn_id, preview, fact_count) console.print(table)` ### Cross-Referencing with Financial Statements `# Find footnotes that reference specific financial statement items def find_footnotes_for_concept(xbrl, concept_name): """Find footnotes related to a specific accounting concept.""" related_footnotes = [] # Find facts matching the concept for fact_key, fact in xbrl.parser.facts.items(): if concept_name.lower() in fact.element_id.lower(): if fact.footnotes: for fn_id in fact.footnotes: if fn_id in xbrl.footnotes: related_footnotes.append((fact, xbrl.footnotes[fn_id])) return related_footnotes # Example: Find footnotes about revenue revenue_footnotes = find_footnotes_for_concept(xbrl, "Revenue") for fact, footnote in revenue_footnotes: print(f"Revenue fact {fact.fact_id}: {footnote.text}")` XBRL Technical Details ---------------------- ### Footnote Structure in XBRL XBRL footnotes follow the XBRL 2.1 specification: ` Explanatory text about the fact. ` ### Supported Footnote Formats EdgarTools handles: - **Standard footnotes** with `id` attributes - **XLink footnotes** with `xlink:label` attributes \- **XHTML content** within footnotes (automatically extracts text) - **Multiple languages** via `xml:lang` attributes - **Custom roles** via `xlink:role` attributes ### Namespace Handling The parser correctly handles all standard XBRL namespaces: - `http://www.xbrl.org/2003/linkbase` (link) - `http://www.w3.org/1999/xlink` (xlink) - `http://www.w3.org/1999/xhtml` (xhtml) - `http://www.w3.org/XML/1998/namespace` (xml) Performance Considerations -------------------------- * Footnote extraction adds minimal overhead to XBRL parsing * Footnotes are parsed lazily during instance document processing * Both fact-to-footnote and footnote-to-fact lookups are O(1) operations * Large documents with many footnotes are handled efficiently Error Handling -------------- The parser gracefully handles common footnote issues: `# Parser warnings for missing footnote references # Warning: "Footnote arc references undefined footnote: footnote_123" # Missing footnote definitions are logged but don't cause parsing to fail # Malformed XHTML content is handled with fallback text extraction` Migration from Manual Parsing ----------------------------- If you were previously parsing footnotes manually: `# Before (manual parsing) import xml.etree.ElementTree as ET def extract_footnotes_manually(xml_content): root = ET.fromstring(xml_content) footnotes = {} # ... complex manual parsing logic ... return footnotes # After (using EdgarTools) from edgar.xbrl.parser import XBRLParser parser = XBRLParser() parser.parse_instance_content(xml_content) footnotes = parser.footnotes # Ready to use!` API Reference ------------- ### XBRLParser.footnotes * **Type**: `Dict[str, Footnote]` * **Description**: Dictionary mapping footnote IDs to Footnote objects ### XBRL.footnotes * **Type**: `Dict[str, Footnote]` * **Description**: Property providing access to parser footnotes ### XBRL.get\_footnotes\_for\_fact(fact\_id: str) * **Parameters**: `fact_id` - The ID of the fact to get footnotes for * **Returns**: `List[Footnote]` - List of associated footnotes * **Description**: Retrieves all footnotes linked to a specific fact ### XBRL.get\_facts\_with\_footnotes() * **Returns**: `Dict[str, Fact]` - Dictionary of facts that have footnotes * **Description**: Returns all facts that reference footnotes ### Fact.footnotes * **Type**: `List[str]` * **Description**: List of footnote IDs that reference this fact ### Fact.fact\_id * **Type**: `Optional[str]` * **Description**: Original `id` attribute from the XML element * * * _This feature was implemented to support the XBRL 2.1 specification for footnotes and is compatible with all standard SEC XBRL filings._ Back to top --- # Fund Voting (N-PX) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/npx-data-object-guide/#n-px-parse-mutual-fund-proxy-voting-records) N-PX: Parse Mutual Fund Proxy Voting Records ============================================ Form N-PX is an annual proxy voting record filed by registered investment companies (mutual funds) to report how they voted on proxy matters for securities they held during the 12-month period ending June 30. This guide details all data available from the `NPX` class for building views. * * * Overview -------- | Property | Type | Description | | --- | --- | --- | | Class Name | `NPX` | | | Forms Handled | `N-PX`, `N-PX/A` | | | Module | `edgar.npx` | | | Source Data | XML primary document + XML proxy vote table | | ### Form Type Descriptions | Form | Description | | --- | --- | | `N-PX` | Annual proxy voting record report | | `N-PX/A` | Amendment to proxy voting record report | * * * Basic Metadata -------------- | Property | Type | Description | Example | | --- | --- | --- | --- | | `submission_type` | `str` | Form type | `"N-PX"` | | `period_of_report` | `str` | Reporting period end date | `"2023-06-30"` | | `report_calendar_year` | `str` | Calendar year of report | `"2023"` | | `is_amendment` | `bool` | Whether this is an amendment | `False` | | `amendment_no` | `str` | Amendment number if applicable | `"1"` | | `amendment_type` | `str` | Type of amendment | `None` | * * * Fund Information ---------------- | Property | Type | Description | Example | | --- | --- | --- | --- | | `fund_name` | `str` | Name of the reporting fund | `"Vanguard Index Funds"` | | `cik` | `str` | Central Index Key | `"0000102909"` | | `report_type` | `str` | Type of report | `"FUND VOTING REPORT"` | | `investment_company_type` | `str` | Investment company type | `"N-1A"` | | `year_or_quarter` | `str` | Year or quarter indicator | `"YEAR"` | | `series_count` | `str` | Number of series in filing | `"5"` | ### Report Types | Type | Description | | --- | --- | | `FUND VOTING REPORT` | Standard fund proxy voting report | | `MANAGER VOTING REPORT` | Investment manager voting report | * * * Regulatory Identifiers ---------------------- | Property | Type | Description | Example | | --- | --- | --- | --- | | `lei_number` | `str` | Legal Entity Identifier | `"549300QSHYB96X84H026"` | | `crd_number` | `str` | Central Registration Depository number | `"12345"` | | `filer_sec_file_number` | `str` | SEC file number of filer | `"811-00234"` | | `npx_file_number` | `str` | N-PX specific file number | `"811-00234"` | * * * Contact Information ------------------- ### Fund Address | Property | Type | Description | Example | | --- | --- | --- | --- | | `address` | `str` | Formatted full address | `"100 Vanguard Blvd\nMalvern, PA 19355"` | | `phone_number` | `str` | Fund phone number | `"610-669-1000"` | ### Agent for Service | Property | Type | Description | | --- | --- | --- | | `agent_for_service_name` | `str` | Name of agent for service | | `agent_for_service_address` | `str` | Formatted agent address | | `agent_for_service_address_street1` | `str` | Street line 1 | | `agent_for_service_address_street2` | `str` | Street line 2 | | `agent_for_service_address_city` | `str` | City | | `agent_for_service_address_state_country` | `str` | State/country | | `agent_for_service_address_zip_code` | `str` | ZIP code | ### Contact Person | Property | Type | Description | | --- | --- | --- | | `contact_name` | `str` | Contact person name | | `contact_phone_number` | `str` | Contact phone number | | `contact_email_address` | `str` | Contact email address | * * * Signature Information --------------------- | Property | Type | Description | Example | | --- | --- | --- | --- | | `signer_name` | `str` | Name of person who signed | `"John W. Smith"` | | `signer_title` | `str` | Title of signer | `"Principal Executive Officer"` | | `signature_date` | `str` | Date filing was signed | `"2023-08-28"` | | `tx_printed_signature` | `str` | Printed signature text | `None` | * * * Proxy Voting Data ----------------- ### Primary Access: `proxy_votes` Property The `proxy_votes` property returns a `ProxyVotes` container with all voting records: `npx = filing.obj() votes = npx.proxy_votes # ProxyVotes container # Convert to DataFrame votes_df = votes.to_dataframe() # Get vote count print(f"Total matters voted: {len(votes)}")` ### ProxyVotes Methods | Method | Returns | Description | | --- | --- | --- | | `to_dataframe()` | `pd.DataFrame` | All votes as DataFrame | | `filter_by_issuer(name)` | `ProxyVotes` | Filter by issuer name (case-insensitive partial match) | | `filter_by_vote(how_voted)` | `ProxyVotes` | Filter by vote type (FOR, AGAINST, ABSTAIN, etc.) | | `filter_by_category(category)` | `ProxyVotes` | Filter by vote category | | `against_management()` | `ProxyVotes` | Filter to votes against management recommendation | | `management_alignment_rate()` | `float` | Rate of alignment with management (0.0 to 1.0) | | `summary()` | `pd.DataFrame` | Vote counts by vote type | | `summary_by_category()` | `pd.DataFrame` | Voting patterns by category | ### Filter Examples `# Filter by issuer apple_votes = npx.proxy_votes.filter_by_issuer("APPLE") # Filter by vote type against_votes = npx.proxy_votes.filter_by_vote("AGAINST") # Filter by category climate_votes = npx.proxy_votes.filter_by_category("CLIMATE") # Find dissenting votes dissent = npx.proxy_votes.against_management() print(f"Voted against management {len(dissent)} times") # Calculate alignment rate alignment = npx.proxy_votes.management_alignment_rate() print(f"Aligned with management {alignment:.1%} of the time")` ### Vote Categories Common vote categories in N-PX filings: | Category | Description | | --- | --- | | `DIRECTOR ELECTIONS` | Board of directors elections | | `SECTION 14A SAY-ON-PAY VOTES` | Executive compensation advisory votes | | `AUDIT-RELATED` | Auditor ratification and related | | `COMPENSATION` | Compensation plans and amendments | | `ENVIRONMENT OR CLIMATE` | Environmental and climate proposals | | `CORPORATE GOVERNANCE` | Governance structure changes | | `OTHER` | Other proposal types | * * * DataFrame Columns ----------------- ### `proxy_votes.to_dataframe()` Columns | Column | Type | Description | Example | | --- | --- | --- | --- | | `issuer_name` | `str` | Company name | `"APPLE INC"` | | `meeting_date` | `str` | Shareholder meeting date | `"2023-03-10"` | | `vote_description` | `str` | Description of the matter | `"Elect Director Tim Cook"` | | `total_shares_voted` | `float` | Total shares voted on matter | `1500000.0` | | `shares_on_loan` | `float` | Shares on loan | `0.0` | | `cusip` | `str` | CUSIP identifier | `"037833100"` | | `isin` | `str` | ISIN identifier | `"US0378331005"` | | `figi` | `str` | FIGI identifier | `None` | | `other_vote_description` | `str` | Additional vote description | `None` | | `vote_source` | `str` | Source of vote | `None` | | `vote_series` | `str` | Series information | `None` | | `vote_other_info` | `str` | Additional info | `None` | | `vote_categories` | `str` | Categories (comma-separated) | `"DIRECTOR ELECTIONS"` | | `other_managers` | `str` | Other managers (comma-separated) | `None` | | `how_voted` | `str` | How fund voted | `"FOR"` | | `shares_voted` | `float` | Shares voted in this record | `1500000.0` | | `management_recommendation` | `str` | Management's recommendation | `"FOR"` | ### How Voted Values | Value | Description | | --- | --- | | `FOR` | Voted in favor | | `AGAINST` | Voted against | | `ABSTAIN` | Abstained from voting | | `WITHHOLD` | Withheld vote (typically for directors) | | `NONE` | Did not vote | * * * Analysis Summary Methods ------------------------ ### `summary()` - Vote Type Distribution `summary = npx.proxy_votes.summary() # Returns DataFrame with columns: vote_type, count` ### `summary_by_category()` - Category Analysis `by_category = npx.proxy_votes.summary_by_category()` | Column | Type | Description | | --- | --- | --- | | `category` | `str` | Vote category type | | `total_votes` | `int` | Total vote records | | `for_votes` | `int` | FOR votes | | `against_votes` | `int` | AGAINST votes | | `abstain_votes` | `int` | ABSTAIN votes | | `other_votes` | `int` | Other vote types | | `with_management` | `int` | Aligned with management | | `against_management` | `int` | Dissented from management | * * * Included Managers ----------------- For consolidated filings that include multiple investment managers: | Property | Type | Description | | --- | --- | --- | | `other_included_managers_count` | `str` | Count of other managers | | `included_managers` | `list[IncludedManager]` | List of included managers | ### IncludedManager Object | Property | Type | Description | | --- | --- | --- | | `serial_no` | `str` | Manager sequence number | | `name` | `str` | Manager name | | `form13f_file_number` | `str` | Form 13F file number | | `sec_file_number` | `str` | SEC file number | `for manager in npx.included_managers: print(f"{manager.serial_no}: {manager.name}")` * * * Series and Class Information ---------------------------- For funds with multiple series: | Property | Type | Description | | --- | --- | --- | | `series_count` | `str` | Number of series | | `series_reports` | `list[SeriesReport]` | Series report details | | `report_series_class_infos` | `list[ReportSeriesClassInfo]` | Series/class mappings | ### SeriesReport Object | Property | Type | Description | | --- | --- | --- | | `id_of_series` | `str` | Series identifier | | `name_of_series` | `str` | Series name | | `lei_of_series` | `str` | Series LEI | ### ReportSeriesClassInfo Object | Property | Type | Description | | --- | --- | --- | | `series_id` | `str` | Series identifier | | `class_infos` | `list[ClassInfo]` | List of class identifiers | * * * Administrative Fields --------------------- | Property | Type | Description | | --- | --- | --- | | `confidential_treatment` | `str` | Confidential treatment flag (Y/N) | | `notice_explanation` | `str` | Notice explanation text | | `explanatory_choice` | `str` | Explanatory choice flag | | `registrant_type` | `str` | Registrant type (RMIC, IA, etc.) | | `live_test_flag` | `str` | Live/Test flag | | `de_novo_request_choice` | `str` | De novo request choice | | `conf_denied_expired` | `str` | Confidential treatment denied/expired | * * * Raw Data Access --------------- | Property | Returns | Description | | --- | --- | --- | | `primary_doc` | `PrimaryDoc` | Raw primary document data | | `filing` | `Filing` | Source Filing object | * * * Related ------- * [Fund Entities](https://edgartools.readthedocs.io/en/latest/guides/fund-entity-guide/) -- look up funds by ticker, navigate hierarchies * [Fund Portfolios (N-PORT)](https://edgartools.readthedocs.io/en/latest/guides/nport-data-object-guide/) -- monthly fund portfolio holdings * [Fund Shareholder Reports (N-CSR)](https://edgartools.readthedocs.io/en/latest/guides/fundshareholderreport-data-object-guide/) -- expense ratios and performance * * * Utility Methods --------------- | Method | Returns | Description | | --- | --- | --- | | `to_dataframe()` | `pd.DataFrame` | Filing metadata as single-row DataFrame | | `__str__()` | `str` | String representation | | `__rich__()` | `Rich object` | Rich console rendering | * * * View Design Recommendations --------------------------- ### Primary View Components 1. **Header Section** 2. Fund name (prominent) 3. Form type (N-PX or N-PX/A) 4. Reporting period 5. Total proxy votes count 6. **Summary Cards** 7. Total matters voted 8. Management alignment rate 9. Vote breakdown (FOR/AGAINST/ABSTAIN) 10. **Voting Table** (main content) 11. Sortable by issuer, date, vote type 12. Columns: Issuer, Meeting Date, Description, How Voted, Shares 13. Category badges 14. Highlight votes against management 15. **Analysis Panels** 16. Vote category breakdown chart 17. Management alignment statistics 18. Top issuers by vote count 19. **Filter Controls** 20. By issuer name 21. By vote type (FOR, AGAINST, etc.) 22. By category 23. By management alignment ### Data Priority for Display | Priority | Data | Reason | | --- | --- | --- | | High | Fund name | Primary identifier | | High | Proxy votes table | Core disclosure | | High | Total vote count | Volume indicator | | High | Management alignment rate | Key metric | | Medium | Period of report | Timing context | | Medium | Vote categories | Analysis dimension | | Medium | Against management votes | Stewardship indicator | | Low | Signer details | Administrative | | Low | Series information | Reference data | | Low | Regulatory identifiers | Technical | ### Visual Indicators (Suggested) | Condition | Visual Treatment | | --- | --- | | Amendment (`N-PX/A`) | Yellow "Amendment" badge | | Vote against management | Red highlight | | High alignment rate (>95%) | Green indicator | | Climate/ESG category | Green badge | | Director election | Blue badge | ### Vote Color Coding | How Voted | Color | Meaning | | --- | --- | --- | | `FOR` | Green | Voted in favor | | `AGAINST` | Red | Voted against | | `ABSTAIN` | Gray | Abstained | | `WITHHOLD` | Orange | Withheld vote | * * * Example Data Structure ---------------------- `{ # Metadata "submission_type": "N-PX", "period_of_report": "2023-06-30", "report_calendar_year": "2023", "is_amendment": False, # Fund info "fund_name": "Vanguard Index Funds", "cik": "0000102909", "report_type": "FUND VOTING REPORT", "investment_company_type": "N-1A", # Identifiers "lei_number": "549300QSHYB96X84H026", "npx_file_number": "811-00234", # Signature "signer_name": "John W. Smith", "signer_title": "Principal Executive Officer", "signature_date": "2023-08-28", # Contact "address": "100 Vanguard Blvd\nMalvern, PA 19355", "phone_number": "610-669-1000", # Included managers (for consolidated filings) "other_included_managers_count": "3", "included_managers": [ { "serial_no": "1", "name": "Vanguard Fixed Income Group", "form13f_file_number": "028-12345", "sec_file_number": "801-12345" } ], # Proxy votes "proxy_votes": [ { "issuer_name": "APPLE INC", "cusip": "037833100", "meeting_date": "2023-03-10", "vote_description": "Elect Director Tim Cook", "total_shares_voted": 1500000.0, "shares_on_loan": 0.0, "vote_categories": ["DIRECTOR ELECTIONS"], "vote_records": [ { "how_voted": "FOR", "shares_voted": 1500000.0, "management_recommendation": "FOR" } ] }, { "issuer_name": "MICROSOFT CORP", "cusip": "594918104", "meeting_date": "2023-12-07", "vote_description": "Report on Climate Lobbying", "total_shares_voted": 2000000.0, "shares_on_loan": 50000.0, "vote_categories": ["ENVIRONMENT OR CLIMATE"], "vote_records": [ { "how_voted": "FOR", "shares_voted": 2000000.0, "management_recommendation": "AGAINST" } ] } ], # Flags "proxy_vote_count": 1500 }` * * * Notes for Implementation ------------------------ 1. **Reporting Period**: N-PX filings cover the 12-month period ending June 30. The `period_of_report` will typically be `YYYY-06-30`. 2. **Vote Records**: Each proxy matter can have multiple `vote_records` if the fund voted different ways on behalf of different accounts or series. 3. **Categories**: A single vote can belong to multiple categories. The `vote_categories` column contains comma-separated values. 4. **Security Identifiers**: Not all votes include CUSIP. Some use ISIN or FIGI. Check all three when matching to other data sources. 5. **Management Alignment**: The `management_alignment_rate()` method provides a key stewardship metric. High alignment (>95%) is typical; low alignment may indicate active voting policies. 6. **Against Management Analysis**: Use `against_management()` to identify controversial votes where the fund disagreed with management's recommendation. 7. **Large Filings**: Major fund families (Vanguard, BlackRock, Fidelity) may have thousands of proxy votes. Consider pagination or lazy loading for UI. 8. **Category Analysis**: The `summary_by_category()` method is useful for understanding voting patterns across different proposal types. 9. **Series Complexity**: Large fund families report votes for multiple series (individual funds). The `series_count` and related properties track this. 10. **Data Quality**: Some older filings may have missing fields. Always check for `None` values before displaying. Back to top --- # Fund Voting (N-PX) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/npx-data-object-guide/#n-px-parse-mutual-fund-proxy-voting-records) N-PX: Parse Mutual Fund Proxy Voting Records ============================================ Form N-PX is an annual proxy voting record filed by registered investment companies (mutual funds) to report how they voted on proxy matters for securities they held during the 12-month period ending June 30. This guide details all data available from the `NPX` class for building views. * * * Overview -------- | Property | Type | Description | | --- | --- | --- | | Class Name | `NPX` | | | Forms Handled | `N-PX`, `N-PX/A` | | | Module | `edgar.npx` | | | Source Data | XML primary document + XML proxy vote table | | ### Form Type Descriptions | Form | Description | | --- | --- | | `N-PX` | Annual proxy voting record report | | `N-PX/A` | Amendment to proxy voting record report | * * * Basic Metadata -------------- | Property | Type | Description | Example | | --- | --- | --- | --- | | `submission_type` | `str` | Form type | `"N-PX"` | | `period_of_report` | `str` | Reporting period end date | `"2023-06-30"` | | `report_calendar_year` | `str` | Calendar year of report | `"2023"` | | `is_amendment` | `bool` | Whether this is an amendment | `False` | | `amendment_no` | `str` | Amendment number if applicable | `"1"` | | `amendment_type` | `str` | Type of amendment | `None` | * * * Fund Information ---------------- | Property | Type | Description | Example | | --- | --- | --- | --- | | `fund_name` | `str` | Name of the reporting fund | `"Vanguard Index Funds"` | | `cik` | `str` | Central Index Key | `"0000102909"` | | `report_type` | `str` | Type of report | `"FUND VOTING REPORT"` | | `investment_company_type` | `str` | Investment company type | `"N-1A"` | | `year_or_quarter` | `str` | Year or quarter indicator | `"YEAR"` | | `series_count` | `str` | Number of series in filing | `"5"` | ### Report Types | Type | Description | | --- | --- | | `FUND VOTING REPORT` | Standard fund proxy voting report | | `MANAGER VOTING REPORT` | Investment manager voting report | * * * Regulatory Identifiers ---------------------- | Property | Type | Description | Example | | --- | --- | --- | --- | | `lei_number` | `str` | Legal Entity Identifier | `"549300QSHYB96X84H026"` | | `crd_number` | `str` | Central Registration Depository number | `"12345"` | | `filer_sec_file_number` | `str` | SEC file number of filer | `"811-00234"` | | `npx_file_number` | `str` | N-PX specific file number | `"811-00234"` | * * * Contact Information ------------------- ### Fund Address | Property | Type | Description | Example | | --- | --- | --- | --- | | `address` | `str` | Formatted full address | `"100 Vanguard Blvd\nMalvern, PA 19355"` | | `phone_number` | `str` | Fund phone number | `"610-669-1000"` | ### Agent for Service | Property | Type | Description | | --- | --- | --- | | `agent_for_service_name` | `str` | Name of agent for service | | `agent_for_service_address` | `str` | Formatted agent address | | `agent_for_service_address_street1` | `str` | Street line 1 | | `agent_for_service_address_street2` | `str` | Street line 2 | | `agent_for_service_address_city` | `str` | City | | `agent_for_service_address_state_country` | `str` | State/country | | `agent_for_service_address_zip_code` | `str` | ZIP code | ### Contact Person | Property | Type | Description | | --- | --- | --- | | `contact_name` | `str` | Contact person name | | `contact_phone_number` | `str` | Contact phone number | | `contact_email_address` | `str` | Contact email address | * * * Signature Information --------------------- | Property | Type | Description | Example | | --- | --- | --- | --- | | `signer_name` | `str` | Name of person who signed | `"John W. Smith"` | | `signer_title` | `str` | Title of signer | `"Principal Executive Officer"` | | `signature_date` | `str` | Date filing was signed | `"2023-08-28"` | | `tx_printed_signature` | `str` | Printed signature text | `None` | * * * Proxy Voting Data ----------------- ### Primary Access: `proxy_votes` Property The `proxy_votes` property returns a `ProxyVotes` container with all voting records: `npx = filing.obj() votes = npx.proxy_votes # ProxyVotes container # Convert to DataFrame votes_df = votes.to_dataframe() # Get vote count print(f"Total matters voted: {len(votes)}")` ### ProxyVotes Methods | Method | Returns | Description | | --- | --- | --- | | `to_dataframe()` | `pd.DataFrame` | All votes as DataFrame | | `filter_by_issuer(name)` | `ProxyVotes` | Filter by issuer name (case-insensitive partial match) | | `filter_by_vote(how_voted)` | `ProxyVotes` | Filter by vote type (FOR, AGAINST, ABSTAIN, etc.) | | `filter_by_category(category)` | `ProxyVotes` | Filter by vote category | | `against_management()` | `ProxyVotes` | Filter to votes against management recommendation | | `management_alignment_rate()` | `float` | Rate of alignment with management (0.0 to 1.0) | | `summary()` | `pd.DataFrame` | Vote counts by vote type | | `summary_by_category()` | `pd.DataFrame` | Voting patterns by category | ### Filter Examples `# Filter by issuer apple_votes = npx.proxy_votes.filter_by_issuer("APPLE") # Filter by vote type against_votes = npx.proxy_votes.filter_by_vote("AGAINST") # Filter by category climate_votes = npx.proxy_votes.filter_by_category("CLIMATE") # Find dissenting votes dissent = npx.proxy_votes.against_management() print(f"Voted against management {len(dissent)} times") # Calculate alignment rate alignment = npx.proxy_votes.management_alignment_rate() print(f"Aligned with management {alignment:.1%} of the time")` ### Vote Categories Common vote categories in N-PX filings: | Category | Description | | --- | --- | | `DIRECTOR ELECTIONS` | Board of directors elections | | `SECTION 14A SAY-ON-PAY VOTES` | Executive compensation advisory votes | | `AUDIT-RELATED` | Auditor ratification and related | | `COMPENSATION` | Compensation plans and amendments | | `ENVIRONMENT OR CLIMATE` | Environmental and climate proposals | | `CORPORATE GOVERNANCE` | Governance structure changes | | `OTHER` | Other proposal types | * * * DataFrame Columns ----------------- ### `proxy_votes.to_dataframe()` Columns | Column | Type | Description | Example | | --- | --- | --- | --- | | `issuer_name` | `str` | Company name | `"APPLE INC"` | | `meeting_date` | `str` | Shareholder meeting date | `"2023-03-10"` | | `vote_description` | `str` | Description of the matter | `"Elect Director Tim Cook"` | | `total_shares_voted` | `float` | Total shares voted on matter | `1500000.0` | | `shares_on_loan` | `float` | Shares on loan | `0.0` | | `cusip` | `str` | CUSIP identifier | `"037833100"` | | `isin` | `str` | ISIN identifier | `"US0378331005"` | | `figi` | `str` | FIGI identifier | `None` | | `other_vote_description` | `str` | Additional vote description | `None` | | `vote_source` | `str` | Source of vote | `None` | | `vote_series` | `str` | Series information | `None` | | `vote_other_info` | `str` | Additional info | `None` | | `vote_categories` | `str` | Categories (comma-separated) | `"DIRECTOR ELECTIONS"` | | `other_managers` | `str` | Other managers (comma-separated) | `None` | | `how_voted` | `str` | How fund voted | `"FOR"` | | `shares_voted` | `float` | Shares voted in this record | `1500000.0` | | `management_recommendation` | `str` | Management's recommendation | `"FOR"` | ### How Voted Values | Value | Description | | --- | --- | | `FOR` | Voted in favor | | `AGAINST` | Voted against | | `ABSTAIN` | Abstained from voting | | `WITHHOLD` | Withheld vote (typically for directors) | | `NONE` | Did not vote | * * * Analysis Summary Methods ------------------------ ### `summary()` - Vote Type Distribution `summary = npx.proxy_votes.summary() # Returns DataFrame with columns: vote_type, count` ### `summary_by_category()` - Category Analysis `by_category = npx.proxy_votes.summary_by_category()` | Column | Type | Description | | --- | --- | --- | | `category` | `str` | Vote category type | | `total_votes` | `int` | Total vote records | | `for_votes` | `int` | FOR votes | | `against_votes` | `int` | AGAINST votes | | `abstain_votes` | `int` | ABSTAIN votes | | `other_votes` | `int` | Other vote types | | `with_management` | `int` | Aligned with management | | `against_management` | `int` | Dissented from management | * * * Included Managers ----------------- For consolidated filings that include multiple investment managers: | Property | Type | Description | | --- | --- | --- | | `other_included_managers_count` | `str` | Count of other managers | | `included_managers` | `list[IncludedManager]` | List of included managers | ### IncludedManager Object | Property | Type | Description | | --- | --- | --- | | `serial_no` | `str` | Manager sequence number | | `name` | `str` | Manager name | | `form13f_file_number` | `str` | Form 13F file number | | `sec_file_number` | `str` | SEC file number | `for manager in npx.included_managers: print(f"{manager.serial_no}: {manager.name}")` * * * Series and Class Information ---------------------------- For funds with multiple series: | Property | Type | Description | | --- | --- | --- | | `series_count` | `str` | Number of series | | `series_reports` | `list[SeriesReport]` | Series report details | | `report_series_class_infos` | `list[ReportSeriesClassInfo]` | Series/class mappings | ### SeriesReport Object | Property | Type | Description | | --- | --- | --- | | `id_of_series` | `str` | Series identifier | | `name_of_series` | `str` | Series name | | `lei_of_series` | `str` | Series LEI | ### ReportSeriesClassInfo Object | Property | Type | Description | | --- | --- | --- | | `series_id` | `str` | Series identifier | | `class_infos` | `list[ClassInfo]` | List of class identifiers | * * * Administrative Fields --------------------- | Property | Type | Description | | --- | --- | --- | | `confidential_treatment` | `str` | Confidential treatment flag (Y/N) | | `notice_explanation` | `str` | Notice explanation text | | `explanatory_choice` | `str` | Explanatory choice flag | | `registrant_type` | `str` | Registrant type (RMIC, IA, etc.) | | `live_test_flag` | `str` | Live/Test flag | | `de_novo_request_choice` | `str` | De novo request choice | | `conf_denied_expired` | `str` | Confidential treatment denied/expired | * * * Raw Data Access --------------- | Property | Returns | Description | | --- | --- | --- | | `primary_doc` | `PrimaryDoc` | Raw primary document data | | `filing` | `Filing` | Source Filing object | * * * Related ------- * [Fund Entities](https://edgartools.readthedocs.io/en/stable/guides/fund-entity-guide/) -- look up funds by ticker, navigate hierarchies * [Fund Portfolios (N-PORT)](https://edgartools.readthedocs.io/en/stable/guides/nport-data-object-guide/) -- monthly fund portfolio holdings * [Fund Shareholder Reports (N-CSR)](https://edgartools.readthedocs.io/en/stable/guides/fundshareholderreport-data-object-guide/) -- expense ratios and performance * * * Utility Methods --------------- | Method | Returns | Description | | --- | --- | --- | | `to_dataframe()` | `pd.DataFrame` | Filing metadata as single-row DataFrame | | `__str__()` | `str` | String representation | | `__rich__()` | `Rich object` | Rich console rendering | * * * View Design Recommendations --------------------------- ### Primary View Components 1. **Header Section** 2. Fund name (prominent) 3. Form type (N-PX or N-PX/A) 4. Reporting period 5. Total proxy votes count 6. **Summary Cards** 7. Total matters voted 8. Management alignment rate 9. Vote breakdown (FOR/AGAINST/ABSTAIN) 10. **Voting Table** (main content) 11. Sortable by issuer, date, vote type 12. Columns: Issuer, Meeting Date, Description, How Voted, Shares 13. Category badges 14. Highlight votes against management 15. **Analysis Panels** 16. Vote category breakdown chart 17. Management alignment statistics 18. Top issuers by vote count 19. **Filter Controls** 20. By issuer name 21. By vote type (FOR, AGAINST, etc.) 22. By category 23. By management alignment ### Data Priority for Display | Priority | Data | Reason | | --- | --- | --- | | High | Fund name | Primary identifier | | High | Proxy votes table | Core disclosure | | High | Total vote count | Volume indicator | | High | Management alignment rate | Key metric | | Medium | Period of report | Timing context | | Medium | Vote categories | Analysis dimension | | Medium | Against management votes | Stewardship indicator | | Low | Signer details | Administrative | | Low | Series information | Reference data | | Low | Regulatory identifiers | Technical | ### Visual Indicators (Suggested) | Condition | Visual Treatment | | --- | --- | | Amendment (`N-PX/A`) | Yellow "Amendment" badge | | Vote against management | Red highlight | | High alignment rate (>95%) | Green indicator | | Climate/ESG category | Green badge | | Director election | Blue badge | ### Vote Color Coding | How Voted | Color | Meaning | | --- | --- | --- | | `FOR` | Green | Voted in favor | | `AGAINST` | Red | Voted against | | `ABSTAIN` | Gray | Abstained | | `WITHHOLD` | Orange | Withheld vote | * * * Example Data Structure ---------------------- `{ # Metadata "submission_type": "N-PX", "period_of_report": "2023-06-30", "report_calendar_year": "2023", "is_amendment": False, # Fund info "fund_name": "Vanguard Index Funds", "cik": "0000102909", "report_type": "FUND VOTING REPORT", "investment_company_type": "N-1A", # Identifiers "lei_number": "549300QSHYB96X84H026", "npx_file_number": "811-00234", # Signature "signer_name": "John W. Smith", "signer_title": "Principal Executive Officer", "signature_date": "2023-08-28", # Contact "address": "100 Vanguard Blvd\nMalvern, PA 19355", "phone_number": "610-669-1000", # Included managers (for consolidated filings) "other_included_managers_count": "3", "included_managers": [ { "serial_no": "1", "name": "Vanguard Fixed Income Group", "form13f_file_number": "028-12345", "sec_file_number": "801-12345" } ], # Proxy votes "proxy_votes": [ { "issuer_name": "APPLE INC", "cusip": "037833100", "meeting_date": "2023-03-10", "vote_description": "Elect Director Tim Cook", "total_shares_voted": 1500000.0, "shares_on_loan": 0.0, "vote_categories": ["DIRECTOR ELECTIONS"], "vote_records": [ { "how_voted": "FOR", "shares_voted": 1500000.0, "management_recommendation": "FOR" } ] }, { "issuer_name": "MICROSOFT CORP", "cusip": "594918104", "meeting_date": "2023-12-07", "vote_description": "Report on Climate Lobbying", "total_shares_voted": 2000000.0, "shares_on_loan": 50000.0, "vote_categories": ["ENVIRONMENT OR CLIMATE"], "vote_records": [ { "how_voted": "FOR", "shares_voted": 2000000.0, "management_recommendation": "AGAINST" } ] } ], # Flags "proxy_vote_count": 1500 }` * * * Notes for Implementation ------------------------ 1. **Reporting Period**: N-PX filings cover the 12-month period ending June 30. The `period_of_report` will typically be `YYYY-06-30`. 2. **Vote Records**: Each proxy matter can have multiple `vote_records` if the fund voted different ways on behalf of different accounts or series. 3. **Categories**: A single vote can belong to multiple categories. The `vote_categories` column contains comma-separated values. 4. **Security Identifiers**: Not all votes include CUSIP. Some use ISIN or FIGI. Check all three when matching to other data sources. 5. **Management Alignment**: The `management_alignment_rate()` method provides a key stewardship metric. High alignment (>95%) is typical; low alignment may indicate active voting policies. 6. **Against Management Analysis**: Use `against_management()` to identify controversial votes where the fund disagreed with management's recommendation. 7. **Large Filings**: Major fund families (Vanguard, BlackRock, Fidelity) may have thousands of proxy votes. Consider pagination or lazy loading for UI. 8. **Category Analysis**: The `summary_by_category()` method is useful for understanding voting patterns across different proposal types. 9. **Series Complexity**: Large fund families report votes for multiple series (individual funds). The `series_count` and related properties track this. 10. **Data Quality**: Some older filings may have missing fields. Always check for `None` values before displaying. Back to top --- # Form Types - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/FormType-Quick-Reference/#formtype-quick-reference-guide) FormType Quick Reference Guide ============================== 🚀 **Getting Started** ---------------------- `from edgar import Company from edgar.enums import FormType company = Company("AAPL") # New: IDE autocomplete for form types filings = company.get_filings(form=FormType.ANNUAL_REPORT) # Old: Still works perfectly filings = company.get_filings(form="10-K")` 📋 **All Available FormTypes** ------------------------------ ### **Periodic Reports** `FormType.ANNUAL_REPORT # "10-K" FormType.QUARTERLY_REPORT # "10-Q" FormType.ANNUAL_REPORT_AMENDED # "10-K/A" FormType.QUARTERLY_REPORT_AMENDED # "10-Q/A" FormType.FOREIGN_ANNUAL # "20-F" FormType.CANADIAN_ANNUAL # "40-F" FormType.EMPLOYEE_BENEFIT_PLAN # "11-K"` ### **Current Reports** `FormType.CURRENT_REPORT # "8-K" FormType.FOREIGN_CURRENT_REPORT # "6-K"` ### **Proxy Statements** `FormType.PROXY_STATEMENT # "DEF 14A" FormType.PRELIMINARY_PROXY # "PRE 14A" FormType.ADDITIONAL_PROXY # "DEFA14A" FormType.MERGER_PROXY # "DEFM14A"` ### **Registration Statements** `FormType.REGISTRATION_S1 # "S-1" FormType.REGISTRATION_S3 # "S-3" FormType.REGISTRATION_S4 # "S-4" FormType.REGISTRATION_S8 # "S-8" FormType.FOREIGN_REGISTRATION_F1 # "F-1" FormType.FOREIGN_REGISTRATION_F3 # "F-3" FormType.FOREIGN_REGISTRATION_F4 # "F-4"` ### **Prospectuses** `FormType.PROSPECTUS_424B1 # "424B1" FormType.PROSPECTUS_424B2 # "424B2" FormType.PROSPECTUS_424B3 # "424B3" FormType.PROSPECTUS_424B4 # "424B4" FormType.PROSPECTUS_424B5 # "424B5"` ### **Ownership Reports** `FormType.BENEFICIAL_OWNERSHIP_13D # "SC 13D" FormType.BENEFICIAL_OWNERSHIP_13G # "SC 13G"` ### **Other Important Forms** `FormType.SPECIALIZED_DISCLOSURE # "SD" FormType.ASSET_BACKED_SECURITIES # "ARS" FormType.LATE_10K_NOTICE # "NT 10-K" FormType.LATE_10Q_NOTICE # "NT 10-Q"` 📚 **Form Collections** ----------------------- `from edgar.enums import PERIODIC_FORMS, PROXY_FORMS, REGISTRATION_FORMS # Pre-defined collections for common workflows PERIODIC_FORMS # [10-K, 10-Q, 10-K/A, 10-Q/A] PROXY_FORMS # [DEF 14A, PRE 14A, DEFA14A, DEFM14A] REGISTRATION_FORMS # [S-1, S-3, S-4, S-8]` ⚡ **Usage Examples** -------------------- ### **Basic Usage** `# Annual reports with autocomplete annual_filings = company.get_filings(form=FormType.ANNUAL_REPORT) # Quarterly reports quarterly_filings = company.get_filings(form=FormType.QUARTERLY_REPORT) # Current reports (8-Ks) current_filings = company.get_filings(form=FormType.CURRENT_REPORT)` ### **Combined Filters** `# Recent annual reports filings = company.get_filings( form=FormType.ANNUAL_REPORT, year=[2022, 2023] ) # Proxy statements this year proxies = company.get_filings( form=FormType.PROXY_STATEMENT, year=2023 )` ### **Multiple Form Types** `# Mix FormType and strings filings = company.get_filings(form=[ FormType.ANNUAL_REPORT, FormType.QUARTERLY_REPORT, "8-K" # String still works ]) # Using form collections periodic_filings = company.get_filings(form=PERIODIC_FORMS)` 🛡️ **Error Handling** ---------------------- `# Typos get helpful suggestions try: filings = company.get_filings(form="10k") # Missing hyphen except ValueError as e: print(e) # "Invalid form type '10k'. Use FormType enum for autocomplete..."` 🔄 **Migration Guide** ---------------------- ### **No Breaking Changes** `# ALL existing code works unchanged: company.get_filings(form="10-K") # ✅ Works company.get_filings(form=["10-K", "10-Q"]) # ✅ Works company.get_filings(form="8-K", year=2023) # ✅ Works` ### **Gradual Adoption** `# Option 1: Keep using strings filings = company.get_filings(form="10-K") # Option 2: Migrate to FormType for autocomplete filings = company.get_filings(form=FormType.ANNUAL_REPORT) # Option 3: Mix as convenient filings = company.get_filings(form=[FormType.ANNUAL_REPORT, "8-K"])` 💡 **IDE Benefits** ------------------- * **Autocomplete**: Type `FormType.` to see all 31 options * **Documentation**: Hover over enums to see SEC form codes * **Type Safety**: mypy/PyCharm catches invalid form parameters * **Refactoring**: Find all usages of specific form types 🔗 **Links** ------------ * **GitHub Discussion**: [#423 Type Hinting Implementation](https://github.com/dgunning/edgartools/discussions/423) * **Feature Branch**: `feat/strenum-type-hinting` * **Test Files**: Run `python formtype_demo_examples.py` for live examples * * * _Perfect backwards compatibility + modern Python typing = Happy developers! 🎉_ Back to top --- # Getting XBRL Data - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/getting-xbrl/#getting-xbrl-data-from-sec-filings) Getting XBRL Data from SEC Filings ================================== Overview -------- The `edgar.xbrl` module provides a powerful yet user-friendly API for processing **XBRL (eXtensible Business Reporting Language)** financial data from SEC filings. Key Features ------------ * **Intuitive API**: Access financial statements with simple, readable method calls * **Multi-period Analysis**: Compare financial data across quarters and years with statement stitching * **Standardized Concepts**: View company-specific terms or standardized labels for cross-company comparison * **Rich Rendering**: Display beautifully formatted financial statements in console or notebooks * **Smart Period Selection**: Automatically identify and select relevant periods for meaningful comparisons * **DataFrame Export**: Convert any statement to pandas DataFrames for further analysis Getting Started --------------- You can get the XBRL from a single filing, or stitch together multiple filings. ### Getting XBRL from a single filing For a single filing you can use `filing.xbrl()` to get the XBRL data, and then access the financial and other statements. `from edgar import Company from edgar.xbrl.xbrl import XBRL # Get a company's latest 10-K filing company = Company('AAPL') filing = company.latest("10-K") # Parse XBRL data xb = filing.xbrl() # Access statements through the user-friendly API statements = xb.statements # Display financial statements balance_sheet = statements.balance_sheet() income_statement = statements.income_statement() cash_flow = statements.cashflow_statement()` ### Getting XBRL from multiple filings You can also stitch together multiple filings to create a multi-period view of financial statements. This uses the `edgar.XBRLS` class to combine data across multiple filings. Each filing should be of the same type (e.g., all 10-Ks or all 10-Qs) and from the same company. `from edgar import Company from edgar.xbrl import XBRLS # Get multiple filings for trend analysis company = Company('AAPL') filings = company.get_filings(form="10-K").head(3) # Get the last 3 annual reports # Create a stitched view across multiple filings xbrls = XBRLS.from_filings(filings) # Access stitched statements stitched_statements = xbrls.statements # Display multi-period statements income_trend = stitched_statements.income_statement() balance_sheet_trend = stitched_statements.balance_sheet() cashflow_trend = stitched_statements.cashflow_statement() # Use view="detailed" to include dimensional breakdowns across periods income_detailed = stitched_statements.income_statement(view="detailed")` User-Friendly Features ---------------------- ### Simple Statement Access Access common financial statements with intuitive methods: `# Get basic statements balance_sheet = statements.balance_sheet() income_statement = statements.income_statement() cash_flow = statements.cashflow_statement() statement_of_equity = statements.statement_of_equity() # Access any statement by type comprehensive_income = statements["ComprehensiveIncome"]` ### Smart Period Views Choose from intelligent period selection views: `# See available period views period_views = statements.get_period_views("IncomeStatement") for view in period_views: print(f"- {view['name']}: {view['description']}") # Render with specific view annual_comparison = statements.income_statement(period_view="Annual Comparison") quarter_comparison = statements.income_statement(period_view="Quarterly Comparison")` ### Easy Conversion to DataFrames Transform any statement into a pandas DataFrame for further analysis: `# Get DataFrame of income statement df = income_statement.to_dataframe()` Statement Stitching for Trend Analysis -------------------------------------- The XBRLS class combines data from multiple periods with intelligent handling of concept changes: `# Create stitched statements across multiple filings xbrls = XBRLS.from_filings(filings) stitched = xbrls.statements # Get a three-year comparison of income statements income_trend = stitched.income_statement(max_periods=3) # Convert to DataFrame for time series analysis trend_df = income_trend.to_dataframe()` Rendering Options ----------------- The XBRL2 module provides flexible output options for financial statements: `# Display with default styling as Rich tables in console/notebooks print(statements.balance_sheet()) # Show full date ranges for duration periods print(statements.income_statement(show_date_range=True)) # Customize period view print(statements.income_statement(period_view="Annual Comparison")) # Convert to pandas DataFrame for analysis df = statements.to_dataframe("BalanceSheet") # Export the statement to markdown income_statement = statements.income_statement() markdown_text = income_statement.render().to_markdown()` ### Statement Display Options The rendering system offers several customization options: | Option | Description | | --- | --- | | `standard=True` | Add `standard_concept` metadata for cross-company analysis (default). Labels remain as company-reported. | | `standard=False` | Skip standardization metadata entirely | | `show_date_range=True` | Show complete date ranges for duration periods (e.g., "Jan 1 - Mar 31, 2023") | | `show_date_range=False` | Show only end dates for cleaner presentation (default) | | `period_view="Name"` | Select a predefined period view ("Annual Comparison", "Quarterly Comparison", etc.) | | `period_filter="duration_..."` | Filter to a specific period by period key | > **Note**: Labels always show the company's original presentation. The `standard_concept` column maps each line item to a standard category (e.g., "Revenue", "CommonEquity") for filtering and cross-company aggregation. Use `df.groupby('standard_concept').sum()` to aggregate by standard concepts. ### The `RenderedStatement` Class The `render_statement()` function returns a `RenderedStatement` object, which provides multiple output formats: `# Get a rendered statement statement = xbrl.render_statement("BalanceSheet") # Display as Rich table (default) print(statement) # Convert to pandas DataFrame df = statement.to_dataframe() # Export to markdown markdown = statement.to_markdown()` ### Customizing Statement Appearance The rendering engine automatically handles: * Proper monetary formatting with scale indicators (thousands, millions, billions) * Appropriate indentation for statement hierarchy * Formatting of section headers and dimension items * Correct display of share counts and per-share values * Fiscal period indicators in statement titles * Unit notes (e.g., "In millions, except per share data") For stitched multi-period statements, you can control periods, date formatting, and dimensional detail: `# Get 3-year comparison with full date ranges annual_trend = stitched_statements.income_statement( max_periods=3, show_date_range=True ) # Include dimensional breakdowns (e.g., cost by segment across years) detailed_trend = stitched_statements.income_statement(view="detailed")` Advanced Features ----------------- ### Custom Period Selection `# Get specific periods from available options available_periods = xbrl.reporting_periods latest_period = available_periods[0] # Render with specific period if latest_period['type'] == 'instant': period_filter = f"instant_{latest_period['date']}" latest_balance_sheet = statements.balance_sheet().render(period_filter=period_filter)` ### Statement Data Exploration `# Get raw statement data for custom processing raw_data = statements.balance_sheet().get_raw_data() # Extract specific information assets = [item for item in raw_data if 'assets' in item['label'].lower()]` Design Philosophy ----------------- The XBRL2 module is designed with these principles: 1. **User-First API**: Simple methods that match how financial analysts think about statements 2. **Intelligent Defaults**: Smart period selection and formatting that "just works" out of the box 3. **Flexible Output Options**: Rich tables for display, DataFrames for analysis, and raw data for custom processing 4. **Consistency Across Companies**: Standardized concepts that enable cross-company comparison Period Selection Logic ---------------------- The XBRL2 module implements sophisticated period selection logic to ensure appropriate periods are displayed for financial statements: ### Quarterly Statement Period Selection When rendering quarterly statements (when fiscal\_period\_focus is Q1, Q2, Q3, or Q4): 1. The system identifies true quarterly periods by filtering duration periods to those with 80-100 day durations 2. If quarterly periods are found, the most recent one is selected as the current quarter 3. For comparison, the system looks for periods with similar duration from approximately 1-2 years prior 4. If no quarterly periods are found, it falls back to the most recent period with a warning ### Annual Statement Period Selection For annual reports (when fiscal\_period\_focus is FY): 1. Annual periods are identified by looking for ~365 day durations or fiscal year markers 2. The system prioritizes periods that align with the entity's fiscal year end 3. Up to three most recent fiscal years are displayed in chronological order This intelligent period selection ensures appropriate periods are displayed for statements, with robust fallbacks when ideal periods aren't available. Enhanced Facts API ------------------ The XBRL2 module includes a powerful facts query interface for direct access to individual XBRL facts: `from edgar import Company from edgar.xbrl import XBRL # Parse XBRL data company = Company('AAPL') filing = company.get_filings(form='10-K').latest() xbrl = XBRL.from_filing(filing) # Access the facts view facts = xbrl.facts # Query facts by various attributes revenue = facts.query().by_concept('Revenue').to_dataframe() balance_sheet_facts = facts.query().by_statement_type('BalanceSheet').to_dataframe() # Use predefined period views - returns important metadata including available periods income_views = facts.get_available_period_views('IncomeStatement') for view in income_views: print(f"- {view['name']}: {view['description']} ({view['facts_count']} facts)") # Get facts filtered by period view annual_comparison = facts.get_facts_by_period_view('IncomeStatement', 'Annual Comparison') # Flexible text search across all text fields (concept, label, element name) earnings_facts = facts.search_facts("Earnings Per Share") # Filter by period keys - useful for custom period selection facts.query().by_period_keys(['duration_2023-01-01_2023-12-31', 'duration_2022-01-01_2022-12-31']).to_dataframe() # Query dimensional data facts_by_segment = facts.query().by_dimension('Segment').to_dataframe() # Safe numeric value filtering with proper None handling large_income_items = (facts.query() .by_statement_type('IncomeStatement') .by_value(lambda v: v > 1_000_000_000) .sort_by('numeric_value', ascending=False) .to_dataframe()) # Time series analysis revenue_over_time = facts.time_series('Revenue')` XBRL Calculation Support ------------------------ The XBRL2 module properly handles calculation relationships from XBRL calculation linkbases: `# Values are automatically adjusted according to calculation weights # For example, elements with negative weights (-1.0) like "IncreaseDecreaseInInventories" # are automatically negated to maintain proper calculation relationships cash_flow_statement = statements.cashflow_statement() # The calculation trees are accessible for inspection for role_uri, calc_tree in xbrl.calculation_trees.items(): print(f"Calculation tree: {calc_tree.definition}") for element_id, node in calc_tree.all_nodes.items(): if node.weight != 1.0: print(f"- {element_id}: weight={node.weight}")` The parser intelligently applies calculation weights to ensure consistent financial data presentation: 1. **Expense Concept Consistency**: Major expense categories (R&D, SG&A, Marketing, etc.) are consistently positive across companies, matching SEC CompanyFacts API behavior 2. **Cash Flow Integrity**: Elements with negative weights (-1.0) in cash flow statements maintain proper sign relationships for accurate calculations 3. **Legitimate Negatives Preserved**: Concepts that should be negative (tax benefits, foreign exchange gains/losses) retain their intended signs 4. **Cross-Company Comparability**: Eliminates inconsistencies where MSFT showed R&D as negative while AAPL showed positive values `# Example: R&D expenses are now consistently positive across companies msft_statements = msft_xbrl.statements.income_statement() aapl_statements = aapl_xbrl.statements.income_statement() # Both show R&D as positive values for proper comparison msft_rnd = msft_statements.get_concept_value("ResearchAndDevelopmentExpense") # $32.5B (positive) aapl_rnd = aapl_statements.get_concept_value("ResearchAndDevelopmentExpense") # $31.4B (positive)` Need help building an XBRL pipeline? The code above extracts XBRL data for one company. Scaling to thousands — with taxonomy normalization, custom extension mapping, and multi-year consistency — is where it gets hard. * **[XBRL consulting for AI & data teams →](https://www.edgar.tools/consulting/xbrl?utm_source=edgartools-docs&utm_medium=see-live&utm_content=xbrl-consulting) ** * **[See all SEC data consulting services →](https://www.edgar.tools/consulting?utm_source=edgartools-docs&utm_medium=see-live&utm_content=xbrl-consulting) ** From the creator of edgartools. [Book a call →](https://www.edgar.tools/consulting/xbrl?utm_source=edgartools-docs&utm_medium=see-live&utm_content=xbrl-consulting#contact) Future Enhancements ------------------- * Enhanced support for non-standard financial statements * Interactive visualization options * Expanded dimensional analysis capabilities * Automatic footnote association * Financial ratio calculations * Advanced calculation validation and reconciliation Back to top --- # Stock Splits & EPS Normalization - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/stock-splits-eps-normalization/#stock-splits-detect-split-events-and-normalize-per-share-metrics-with-python) Stock Splits: Detect Split Events and Normalize Per-Share Metrics with Python ============================================================================= Stock splits change share counts and per-share values, making historical comparisons difficult. A company reporting $10 EPS before a 2-for-1 split should show $5 adjusted EPS for that period -- but SEC filings contain both adjusted and unadjusted values depending on when they were filed. EdgarTools detects splits from XBRL data and automatically normalizes per-share metrics so you can build consistent time series. `from edgar import Company from edgar.ttm import detect_splits company = Company("NVDA") facts = company.get_facts() splits = detect_splits(facts.get_all_facts()) for split in splits: print(f"{split['date']}: {split['ratio']:.0f}-for-1 split")` A few lines to find every stock split in a company's SEC filing history. * * * Detect Stock Splits from XBRL Data ---------------------------------- EdgarTools finds splits by looking for `StockSplitConversionRatio` facts in XBRL filings. These facts appear in 10-Q, 10-K, and 8-K reports whenever a company reports a split event. `from edgar import Company from edgar.ttm import detect_splits nvidia = Company("NVDA") entity_facts = nvidia.get_facts() facts = entity_facts.get_all_facts() splits = detect_splits(facts) for split in splits: print(f"Date: {split['date']}, Ratio: {split['ratio']}")` Output: `Date: 2021-07-20, Ratio: 4.0 Date: 2024-06-10, Ratio: 10.0` ### How Split Detection Works The `detect_splits()` function: 1. **Finds split facts** - Searches for `StockSplitConversionRatio` in XBRL concepts 2. **Filters stale data** - Rejects facts filed >280 days after the split date (historical echoes) 3. **Validates duration** - Accepts instant facts or short-duration facts (≤31 days), rejects quarterly/annual aggregations 4. **Deduplicates** - One split per year/ratio combination (same split reported in multiple filings) The ratio represents the multiplier applied to share counts. A 10-for-1 split has `ratio=10.0`. * * * Find Stock Split Announcements in 8-K Filings --------------------------------------------- Companies typically announce splits via 8-K current reports, usually under Item 8.01 ("Other Events") or occasionally Item 5.03 ("Amendments to Articles of Incorporation"). `from edgar import Company apple = Company("AAPL") filings_8k = apple.get_filings(form="8-K") # Filter to filings that might contain split announcements for filing in filings_8k[:50]: # Check recent 50 filings eight_k = filing.obj() # Look for Item 8.01 (where most splits are announced) if 'Item 8.01' in eight_k.items or 'Item 5.03' in eight_k.items: content = eight_k.get('8.01') or eight_k.get('5.03') or '' if 'split' in content.lower(): print(f"{filing.filing_date}: {filing.accession_no}") print(f"Items: {', '.join(eight_k.items)}")` Press releases attached as EX-99 exhibits often contain the split announcement details: `if eight_k.has_press_release: for release in eight_k.press_releases: text = release.text() if 'stock split' in text.lower(): print(f"Split announced: {filing.filing_date}") print(text[:500]) # First 500 chars` * * * Normalize Per-Share Metrics --------------------------- Once you've detected splits, use `apply_split_adjustments()` to retroactively adjust historical data. This makes pre-split and post-split values directly comparable. `from edgar import Company from edgar.ttm import detect_splits, apply_split_adjustments tesla = Company("TSLA") facts = tesla.get_facts().get_all_facts() # Detect splits splits = detect_splits(facts) # Apply adjustments adjusted_facts = apply_split_adjustments(facts, splits) # Compare original vs adjusted EPS eps_facts = [f for f in facts if 'EarningsPerShare' in f.concept] eps_adjusted = [f for f in adjusted_facts if 'EarningsPerShare' in f.concept] for orig, adj in zip(eps_facts[:3], eps_adjusted[:3]): print(f"{orig.period_end}: ${orig.numeric_value:.2f} → ${adj.numeric_value:.2f}")` Output shows EPS before and after split adjustment: `2021-12-31: $4.90 → $1.63 2022-03-31: $3.22 → $1.07 2022-06-30: $2.27 → $0.76` ### What Gets Adjusted The adjustment logic depends on the fact's unit and concept: | Type | Unit Pattern | Adjustment | Examples | | --- | --- | --- | --- | | **Per-share metrics** | `/share` in unit or `EarningsPerShare` in concept | Divide by ratio | EPS, Dividends per share, Book value per share | | **Share counts** | `shares` in unit (but not per-share) | Multiply by ratio | Shares outstanding, Weighted average shares | | **Other metrics** | All others | No adjustment | Revenue, Net Income, Assets | `# Per-share: Divide by split ratio # If 10-for-1 split, $10 EPS becomes $1 EPS adjusted_eps = original_eps / 10.0 # Share counts: Multiply by split ratio # If 10-for-1 split, 100M shares becomes 1B shares adjusted_shares = original_shares * 10.0` ### Retroactive Adjustment Rules Splits only adjust facts from periods **before** the split date. The function applies cumulative ratios for multiple splits: `# Example: Company had two splits # 2021-07-20: 4-for-1 split # 2024-06-10: 10-for-1 split # For a fact from 2020: # - Both splits occurred after 2020 # - Cumulative ratio = 4.0 * 10.0 = 40.0 # - Adjust by dividing by 40 # For a fact from 2022: # - Only 2024 split occurred after 2022 # - Cumulative ratio = 10.0 # - Adjust by dividing by 10 # For a fact from 2024-07: # - No splits after this date # - No adjustment needed (already post-split values)` The function also checks filing dates. If a fact was filed **after** the split, it's already adjusted by the company and doesn't need further modification. * * * Automatic Split Handling in TTM Calculations -------------------------------------------- TTM (Trailing Twelve Months) methods automatically detect and apply split adjustments. You don't need to call `detect_splits()` or `apply_split_adjustments()` manually when using these methods. `from edgar import Company nvidia = Company("NVDA") # TTM calculations handle splits automatically ttm_revenue = nvidia.get_ttm_revenue() ttm_net_income = nvidia.get_ttm_net_income() print(f"TTM Revenue: ${ttm_revenue.value / 1e9:.1f}B") print(f"TTM Net Income: ${ttm_net_income.value / 1e9:.1f}B") print(f"As of: {ttm_revenue.as_of_date}") print(f"Periods: {ttm_revenue.periods}")` For any XBRL concept: `# Per-share metrics are automatically split-adjusted ttm_eps = nvidia.get_ttm("EarningsPerShareBasic") print(f"TTM EPS: ${ttm_eps.value:.2f}") # Share counts are automatically adjusted too ttm_shares = nvidia.get_ttm("WeightedAverageNumberOfSharesOutstandingBasic") print(f"Weighted Avg Shares: {ttm_shares.value / 1e9:.2f}B")` Behind the scenes, `get_ttm()` calls the internal method `_get_split_adjusted_facts()` which: 1. Gets all facts for the company 2. Detects splits using `detect_splits()` 3. Applies adjustments using `apply_split_adjustments()` 4. Returns normalized facts for TTM calculation * * * Complete Workflow: NVIDIA 10-for-1 Split Example ------------------------------------------------ NVIDIA executed a 10-for-1 stock split on June 10, 2024. Let's build a complete workflow that detects this split, normalizes historical EPS, and validates the adjustments. `from edgar import Company from edgar.ttm import detect_splits, apply_split_adjustments # Step 1: Get NVIDIA data nvidia = Company("NVDA") facts = nvidia.get_facts().get_all_facts() # Step 2: Detect splits splits = detect_splits(facts) print(f"Found {len(splits)} splits:") for split in splits: print(f" {split['date']}: {split['ratio']:.0f}-for-1") # Step 3: Filter to EPS facts eps_facts = [f for f in facts if 'EarningsPerShareBasic' in f.concept and f.period_end and f.numeric_value is not None] # Sort by period eps_facts.sort(key=lambda f: f.period_end) # Step 4: Apply split adjustments adjusted_facts = apply_split_adjustments(eps_facts, splits) # Step 5: Compare pre-split periods print("\nEPS Comparison (split-adjusted):") print(f"{'Period':<12} {'Original':>12} {'Adjusted':>12} {'Cumulative Ratio':>18}") print("-" * 58) for orig, adj in zip(eps_facts[-8:], adjusted_facts[-8:]): # Calculate cumulative ratio from the adjustment context context = adj.calculation_context or "" if "split_adj_ratio" in context: ratio = float(context.split("_")[-1]) else: ratio = 1.0 print(f"{orig.period_end!s:<12} ${orig.numeric_value:>11.2f} " f"${adj.numeric_value:>11.2f} {ratio:>17.1f}x")` Expected output: `Found 2 splits: 2021-07-20: 4-for-1 2024-06-10: 10-for-1 EPS Comparison (split-adjusted): Period Original Adjusted Cumulative Ratio ---------------------------------------------------------- 2022-01-30 $ 4.44 $ 0.11 40.0x 2022-05-01 $ 1.36 $ 0.03 40.0x 2022-07-31 $ 0.51 $ 0.01 40.0x 2022-10-30 $ 0.58 $ 0.01 40.0x 2023-01-29 $ 0.88 $ 0.02 40.0x 2023-04-30 $ 1.09 $ 0.03 40.0x 2023-07-30 $ 2.70 $ 0.07 40.0x 2024-01-28 $ 5.16 $ 0.13 40.0x` Periods before both splits get cumulative adjustment of 40x (4 × 10). Periods between the splits would get 10x adjustment. Periods after June 2024 need no adjustment. * * * Common Analysis Patterns ------------------------ ### Track Split History Across Multiple Companies `from edgar import Company from edgar.ttm import detect_splits tech_stocks = ["AAPL", "MSFT", "GOOGL", "AMZN", "NVDA", "TSLA"] for ticker in tech_stocks: try: company = Company(ticker) facts = company.get_facts() splits = detect_splits(facts.get_all_facts()) if splits: print(f"\n{ticker} ({company.name}):") for split in splits: print(f" {split['date']}: {split['ratio']:.1f}-for-1") else: print(f"\n{ticker}: No splits detected") except Exception as e: print(f"\n{ticker}: Error - {e}")` ### Build Split-Adjusted Time Series Compare EPS over multiple years with consistent per-share values: `from edgar import Company from edgar.ttm import detect_splits, apply_split_adjustments import pandas as pd company = Company("AAPL") facts = company.get_facts().get_all_facts() # Get splits and adjust facts splits = detect_splits(facts) adjusted_facts = apply_split_adjustments(facts, splits) # Extract EPS time series eps_data = [] for f in adjusted_facts: if 'EarningsPerShareBasic' in f.concept and f.fiscal_period == 'FY': eps_data.append({ 'fiscal_year': f.fiscal_year, 'period_end': f.period_end, 'eps': f.numeric_value }) # Create DataFrame df = pd.DataFrame(eps_data).sort_values('fiscal_year') print(df) # Calculate growth rates on split-adjusted data df['yoy_growth'] = df['eps'].pct_change() * 100 print(f"\nAverage EPS growth: {df['yoy_growth'].mean():.1f}%")` ### Validate Split Adjustments Against Company Reports Companies publish adjusted historical data after splits. You can validate the adjustment logic: `from edgar import Company from edgar.ttm import detect_splits, apply_split_adjustments company = Company("NVDA") facts = company.get_facts().get_all_facts() splits = detect_splits(facts) adjusted = apply_split_adjustments(facts, splits) # Find the same period reported before and after split # Pre-split: Filed in 2023 10-K, reports 2023 EPS (pre-split basis) # Post-split: Filed in 2024 10-K, reports 2023 EPS (post-split basis) eps_2023_filings = [f for f in adjusted if 'EarningsPerShareBasic' in f.concept and f.fiscal_year == 2023] # Group by filing date from collections import defaultdict by_filing = defaultdict(list) for f in eps_2023_filings: by_filing[f.filing_date].append(f) # Compare values across filings for filing_date in sorted(by_filing.keys())[:3]: facts_list = by_filing[filing_date] if facts_list: f = facts_list[0] print(f"Filed {filing_date}: 2023 EPS = ${f.numeric_value:.2f}")` ### Calculate Split-Adjusted Market Cap History Combine share counts and stock prices to build historical market cap: `from edgar import Company from edgar.ttm import detect_splits, apply_split_adjustments company = Company("AAPL") facts = company.get_facts().get_all_facts() splits = detect_splits(facts) adjusted = apply_split_adjustments(facts, splits) # Get split-adjusted shares outstanding shares_facts = [f for f in adjusted if 'CommonStockSharesOutstanding' in f.concept and f.fiscal_period == 'FY'] for f in sorted(shares_facts, key=lambda x: x.period_end)[-5:]: shares_b = f.numeric_value / 1e9 print(f"{f.period_end}: {shares_b:.2f}B shares (split-adjusted)")` * * * API Quick Reference ------------------- ### Detection and Adjustment Functions | Function | Parameters | Returns | Description | | --- | --- | --- | --- | | `detect_splits(facts)` | `facts`: List of FinancialFact | List of dicts with `date` and `ratio` | Find all stock splits in facts | | `apply_split_adjustments(facts, splits)` | `facts`: List of FinancialFact
`splits`: List of split dicts | List of adjusted FinancialFact | Apply retroactive adjustments | ### Company Methods with Automatic Split Handling | Method | Returns | Description | | --- | --- | --- | | `company.get_ttm(concept, as_of=None)` | `TTMMetric` | TTM for any concept (split-adjusted) | | `company.get_ttm_revenue(as_of=None)` | `TTMMetric` | TTM revenue (split-adjusted) | | `company.get_ttm_net_income(as_of=None)` | `TTMMetric` | TTM net income (split-adjusted) | ### Split Detection Parameters The detection logic uses these constants from `edgar.ttm.calculator`: | Constant | Value | Purpose | | --- | --- | --- | | `MAX_SPLIT_LAG_DAYS` | 280 | Maximum days between split date and filing date | | `MAX_SPLIT_DURATION_DAYS` | 31 | Maximum period duration for split facts | * * * Things to Know -------------- **Splits apply retroactively to historical data only.** Facts from periods after the split date don't need adjustment -- they're already reported on a post-split basis. **Filing date matters for restated facts.** If a fact was filed after a split date, the company has already adjusted it. The adjustment logic checks `filing_date` to avoid double-adjusting. **Multiple splits compound.** A company with a 4-for-1 split in 2021 and a 10-for-1 split in 2024 requires a cumulative adjustment of 40x for pre-2021 data. **Reverse splits work automatically.** A 1-for-10 reverse split has `ratio=0.1`. Per-share metrics get divided by 0.1 (multiplied by 10), which correctly increases the adjusted historical EPS. **Balance sheet items don't need adjustment.** Assets, liabilities, and equity are not per-share values. Total stockholders' equity stays the same regardless of share count. **Not all facts have filing dates.** The adjustment logic handles `None` filing dates by assuming the fact needs adjustment if it's from before the split. **Instant facts are preferred.** Split events are moment-in-time occurrences. The detector accepts instant facts (no `period_start`) or short-duration facts (≤31 days) but rejects quarterly/annual durations. **8-K filing timing varies.** While Item 8.01 is common for split announcements, check Items 5.03 and exhibit press releases. Not all companies follow the same disclosure pattern. **Weighted average share counts need special handling.** These represent time-weighted averages over a period. For Q4 EPS derivation with splits, use the formula: Q4 shares = 4 × annual\_shares - 3 × YTD\_9M\_shares. * * * Related ------- * [Financial Data](https://edgartools.readthedocs.io/en/stable/guides/financial-data/) - Extract financial statements and metrics * [8-K Current Reports](https://edgartools.readthedocs.io/en/stable/eightk-filings/) - Parse material event filings * [Company Facts API](https://edgartools.readthedocs.io/en/stable/guides/company-facts/) - Access XBRL facts programmatically Back to top --- # Dimensions - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/xbrl/concepts/dimension-handling/#understanding-dimensions-in-financial-statements) Understanding Dimensions in Financial Statements ================================================ This guide explains how EdgarTools handles XBRL dimensions and how to get complete, accurate financial statements. Quick Summary ------------- When you retrieve a financial statement, EdgarTools automatically: - **Shows** values that belong on the face of the statement (including dimensional face values) - **Hides** breakdown details that belong in notes disclosures (geographic, segment, etc.) `from edgar import Company company = Company("WDAY") filing = company.get_filings(form="10-K").latest() xbrl = filing.xbrl() # Default: Shows face presentation (what you'd see in SEC Viewer) income = xbrl.statements.income_statement() print(income) # Full data: Shows everything for custom analysis df = income.to_dataframe(view="detailed")` Why Dimensions Matter --------------------- Many companies report financial values **only through dimensional XBRL**. Without proper handling, these statements appear incomplete or out-of-balance. ### The Scale of the Problem Based on community research ([GH-577](https://github.com/dgunning/edgartools/issues/577) ), dimensional-only reporting is widespread: **Income Statement - Cost of Goods Sold:** - BA (Boeing) 2023+, CARR (Carrier) 2020+, GD (General Dynamics) 2020+ - HII (Huntington Ingalls) 2020+, INTU (Intuit) 2018+, NOC (Northrop Grumman) 2019+ - RTX 2019+, SLB (Schlumberger) 2018+, WDAY (Workday) 2019+ - CHH 2022+, CHRW 2018+, CTAS 2020+, GEHC 2024+, MAR 2022+ - OTIS 2020+, PFE 2022, TT 2021+, UPS 2018, VZ 2020+ **Balance Sheet - Various Line Items:** - Goodwill: BSX 2019, IBM 2023+, JKHY 2016+, MCD 2023 - PPE: BSX 2019, CSX 2015+, HLT 2022+, PFE 2022 - Receivables: COP 2023+, FIS 2024+, GEHC 2023+, LYB 2023+ - Payables: COP 2023+, HLT 2019+, LYB 2023+, WDC 2023+ - Debt: ADP 2023+, CAT 2020+, HLT 2019+ - Contract Liabilities: BBY 2021+, HLT 2019+, MAR 2018+, REGN 2020 ### Example: Workday Income Statement Workday reports Cost of Goods Sold exclusively via `ProductOrServiceAxis`: `2025 2024 2023 Revenue: Subscription services $7,718M $6,603M $5,567M Professional services $728M $656M $649M Total Revenue $8,446M $7,259M $6,216M Cost of Goods Sold: Subscription services $1,266M $1,031M $1,007M Professional services $803M $740M $703M Total COGS $2,069M $1,771M $1,710M` **Without dimensional handling**: COGS would show as NaN/missing, making it impossible to calculate gross margin. **With EdgarTools**: Both subscription and professional services COGS values are preserved, and the statement balances correctly. ### Example: Caterpillar Balance Sheet Caterpillar reports debt through dimensional XBRL across multiple years: | Year | Concept | Axis Used | | --- | --- | --- | | 2020-2025 | ShortTermBorrowings | ConsolidationItemsAxis | | 2020-2025 | LongTermDebtCurrent | ConsolidationItemsAxis | | 2020-2025 | LongTermDebt | ConsolidationItemsAxis | EdgarTools preserves these values so debt totals appear correctly on the balance sheet. How It Works ------------ ### Face Values vs Breakdowns Not all dimensional data is the same: | Type | Description | Example | Shown by Default? | | --- | --- | --- | --- | | **Face Value** | Values that appear on the statement face | Product vs Service revenue | ✅ Yes | | **Breakdown** | Drill-down details for notes disclosures | Revenue by country | ❌ No | > **Member Hierarchy (v5.21.1+):** When using `view="detailed"`, sub-members within a dimension are now properly nested under their parent members. For example, Tesla's "Automotive sales" and "Automotive regulatory credits" appear as children of "Automotive Revenues" with increasing `level` values, reflecting the definition linkbase hierarchy. ### Classification Logic EdgarTools uses a tiered approach to classify dimensions: **Tier 1: Definition Linkbase (Authoritative)** - The XBRL filing itself declares which dimensions are valid for each statement - If declared in the definition linkbase, it's a face value - Highest confidence classification **Tier 2: Curated Axis Lists** - Known face-level axes: `ProductOrServiceAxis`, `DebtInstrumentAxis`, `PropertyPlantAndEquipmentByTypeAxis` - Known breakdown axes: `StatementGeographicalAxis`, `StatementBusinessSegmentsAxis`, `BusinessAcquisitionAxis` - Based on empirical analysis of S&P 500 filings **Tier 3: Pattern Matching** - Axes matching patterns like `FairValue*Axis` or `*HierarchyLevelAxis` are classified as breakdowns - Fallback when other methods don't apply Usage Guide ----------- ### Standard View (Default) Get the statement as it would appear in the SEC Viewer: `from edgar import Company company = Company("SLB") filing = company.get_filings(form="10-K").latest() xbrl = filing.xbrl() # Face presentation - includes dimensional face values income = xbrl.statements.income_statement() print(income)` Output shows COGS by Product and Services (the dimensional face values): `Cost of Goods and Services Sold: Product $10,982M Services $17,847M` ### Full Data View Get all data including breakdowns for custom analysis: `# All dimensional data included — use view="detailed" df = income.to_dataframe(view="detailed") # Filter as needed for your analysis geographic_breakdown = df[df['dimension_label'].str.contains('Geographic', na=False)]` ### Working with Dimensional Data The dataframe includes helpful columns for understanding dimensions: `df = income.to_dataframe(view="detailed") # Key columns: # - 'dimension': True/False - is this a dimensional row? # - 'is_breakdown': True/False - is this a breakdown (vs face value)? # - 'dimension_label': Human-readable dimension info # Find all face-level dimensional values face_dimensional = df[(df['dimension'] == True) & (df['is_breakdown'] == False)] # Find all breakdown values breakdowns = df[df['is_breakdown'] == True]` ### Calculating Totals When a concept has dimensional values but no non-dimensional total, you may need to sum: `df = income.to_dataframe(view="standard") # COGS may have individual values but NaN for total cogs_rows = df[df['concept'] == 'us-gaap_CostOfGoodsAndServicesSold'] # Sum the non-NaN values for the total period_col = '2025-01-31' # or whichever period you need total_cogs = cogs_rows[period_col].sum()` Dimension Classification Reference ---------------------------------- ### Face-Level Axes (Always Shown) These dimensions represent valid face presentation and are preserved by default: | Axis | Usage | | --- | --- | | `ProductOrServiceAxis` | Product vs Service breakdown (revenue, COGS) | | `PropertyPlantAndEquipmentByTypeAxis` | PPE categories | | `DebtInstrumentAxis` | Debt instrument types | | `LongtermDebtTypeAxis` | Long-term debt categories | | `ShortTermDebtTypeAxis` | Short-term debt categories | | `StatementClassOfStockAxis` | Stock class distinctions | | `ContracttypeAxis` | Contract types (defense contractors) | | `MajorProgramsAxis` | Major program breakdown (defense) | ### Breakdown Axes (Filtered by Default) These dimensions represent notes disclosures and are hidden by default: | Axis | Usage | | --- | --- | | `StatementGeographicalAxis` | Geographic segment breakdown | | `StatementBusinessSegmentsAxis` | Business segment breakdown | | `BusinessAcquisitionAxis` | Acquisition-specific details | | `ConsolidationItemsAxis` | Consolidation eliminations | | `MajorCustomersAxis` | Customer concentration | | `RestatementAxis` | Prior period adjustments | | `FairValueByFairValueHierarchyLevelAxis` | Fair value hierarchy | | `RetirementPlanTypeAxis` | Pension plan details | ### Context-Dependent Axes Some axes behave differently based on statement type: `# StatementEquityComponentsAxis: # - On Statement of Equity: STRUCTURAL (defines columns) - shown # - On Balance Sheet: BREAKDOWN (notes detail) - hidden` Troubleshooting --------------- ### Statement Shows NaN for Expected Values **Possible causes:** 1. **Dimensional-only value with old EdgarTools version**: Upgrade to v5.7.4+ 2. **No total row exists**: The XBRL only has dimensional breakdown, no aggregated total 3. **Unknown axis**: The dimension axis isn't in our classification lists **Solution:** `# Check what's in the full data df = statement.to_dataframe(view="detailed") concept_rows = df[df['concept'].str.contains('YourConcept')] print(concept_rows[['label', 'dimension', 'dimension_label', value_column]])` ### Statement Doesn't Balance **Check the dimensional data:** `df = income.to_dataframe(view="detailed") # Look for missing values that might be dimensional missing = df[df[value_column].isna() & (df['abstract'] == False)] print(missing[['concept', 'label', 'dimension']])` ### Need a Specific Breakdown `# Get all data first df = statement.to_dataframe(view="detailed") # Filter to specific dimension geographic = df[df['dimension_label'].str.contains('Geographic', na=False)]` API Reference ------------- ### Statement Methods `# Get statement with default handling (face values preserved) statement = xbrl.statements.income_statement() # Control dimensional data with the view parameter df = statement.to_dataframe(view="standard") # Face presentation (default for display) df = statement.to_dataframe(view="detailed") # All dimensional data included df = statement.to_dataframe(view="summary") # Non-dimensional totals only # The view parameter also works on stitched (multi-period) statements income = xbrls.statements.income_statement(view="detailed") df = income.to_dataframe()` The `view` parameter accepts `"standard"`, `"detailed"`, or `"summary"` (or the `StatementView` enum). The legacy `include_dimensions` boolean is still supported but `view` is preferred. ### Dimension Classification API `from edgar.xbrl.dimensions import ( classify_dimension_with_confidence, DimensionConfidence, is_breakdown_dimension, ) # Check if an item is a breakdown is_breakdown = is_breakdown_dimension(item, xbrl=xbrl, role_uri=role_uri) # Get detailed classification classification, confidence, reason = classify_dimension_with_confidence( item, xbrl=xbrl, role_uri=role_uri ) # Returns: ('face', DimensionConfidence.HIGH, 'Declared in definition linkbase')` ### XBRL Dimension Methods `# Check if definition linkbase exists for a role has_def = xbrl.has_definition_linkbase_for_role(role_uri) # Check if a specific dimension is valid for a role is_valid = xbrl.is_dimension_valid_for_role('srt:ProductOrServiceAxis', role_uri) # Get all valid dimensions for a role valid_dims = xbrl.get_valid_dimensions_for_role(role_uri)` Version History --------------- | Version | Change | | --- | --- | | v5.21.1 | Member hierarchy support — sub-members nested under parent members in `to_dataframe(view="detailed")` using definition linkbase hierarchy | | v5.7.4 | Definition linkbase-based dimension filtering (GH-577 fix) | | v5.7.2 | Initial dimension handling with hardcoded lists (GH-569) | | v5.7.0 | Changed default to `include_dimensions=False` | Related Resources ----------------- * [XBRL Documentation Hub](https://edgartools.readthedocs.io/en/stable/xbrl/) - Central navigation for all XBRL docs * [Extract Financial Statements Guide](https://edgartools.readthedocs.io/en/stable/guides/extract-statements/) - Complete guide to extracting financial data * [XBRL Standardization Concepts](https://edgartools.readthedocs.io/en/stable/xbrl/concepts/standardization/) - 95 standard concepts for cross-company comparison * [Multi-Period Analysis](https://edgartools.readthedocs.io/en/stable/xbrl/guides/multi-period-analysis/) - Working with multiple filings * [GitHub Issue #577](https://github.com/dgunning/edgartools/issues/577) - Original problem documentation * [SEC Financial Statement Data Sets](https://www.sec.gov/data-research/sec-markets-data/financial-statement-notes-data-sets) - SEC's processed XBRL data Acknowledgments --------------- Special thanks to [@mpreiss9](https://github.com/mpreiss9) for extensive research documenting dimensional-only reporting patterns across hundreds of filings, which directly informed this implementation. Back to top --- # SEC Rate Limits & Compliance - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/resources/sec-compliance/#sec-rate-limits-compliance) SEC Rate Limits & Compliance ============================ The SEC EDGAR system is a valuable public resource that provides access to corporate filings. To ensure fair access for all users, the SEC has established guidelines and rate limits for automated access. This guide explains these requirements and how to use edgartools in a compliant manner. SEC EDGAR Access Requirements ----------------------------- ### Fair Access Policy The SEC maintains a [Fair Access Policy](https://www.sec.gov/os/accessing-edgar-data) that requires all automated EDGAR access to: 1. Identify the accessing user/organization in the HTTP request 2. Limit request rates to avoid overloading the system 3. Respect the `robots.txt` directives 4. Access data during appropriate hours ### Required Identity Information When using automated tools to access EDGAR, you must identify yourself by providing: * Your name or organization name * Your email address This allows the SEC to contact you if there are issues with your access patterns. Setting Your Identity in edgartools ----------------------------------- edgartools makes it easy to comply with SEC requirements by providing a simple way to set your identity: `from edgar import set_identity # Set your identity information set_identity( name="Your Name", email="your.email@example.com", organization="Your Organization" # Optional )` This identity information will be included in the `User-Agent` header of all requests made by edgartools. ### Default Behavior If you don't explicitly set your identity, edgartools will: 1. Look for environment variables `EDGAR_NAME` and `EDGAR_EMAIL` 2. If not found, use a generic identity that indicates edgartools usage However, it's strongly recommended to set your own identity to ensure compliance with SEC requirements. Understanding SEC Rate Limits ----------------------------- The SEC doesn't publish specific rate limits, but based on their guidelines and observed behavior, the following limits are recommended: * No more than 10 requests per second * Reasonable total volume per day * Avoid excessive concurrent requests ### edgartools Default Rate Limiting By default, edgartools implements conservative rate limiting: * Maximum of 10 requests per second * Built-in delays between requests * Automatic retries with exponential backoff for 429 errors This default configuration is designed to keep you compliant with SEC guidelines while still providing good performance. Customizing Rate Limits ----------------------- You can adjust the rate limits in edgartools if needed: `from edgar import set_rate_limit # Set a more conservative rate limit (requests per second) set_rate_limit(5) # 5 requests per second` For high-volume or production use cases, consider being more conservative with your rate limits to avoid potential IP blocks. Signs of Exceeding Rate Limits ------------------------------ If you exceed SEC rate limits, you may experience: 1. HTTP 429 (Too Many Requests) responses 2. HTTP 403 (Forbidden) responses 3. Temporary IP blocks (typically 10 minutes to 24 hours) edgartools will automatically handle 429 responses with retries, but persistent rate limit violations may result in longer blocks. Best Practices for Compliant Access ----------------------------------- ### 1\. Always Set Your Identity `from edgar import set_identity set_identity( name="Your Name", email="your.email@example.com" )` ### 2\. Use Local Storage Reduce the number of requests by storing filings locally: `from edgar import enable_local_storage enable_local_storage("/path/to/storage")` ### 3\. Implement Appropriate Delays For batch processing, add delays between operations: `import time for filing in filings: # Process filing process_filing(filing) # Add delay between filings time.sleep(0.2) # 200ms delay` ### 4\. Use Efficient Query Patterns Choose the most efficient access pattern for your needs: `# For company-specific queries, use company.get_filings() # (makes just one request for all filings) company = Company("AAPL") filings = company.get_filings(form="10-K") # For form-specific queries across companies, use get_filings() # (makes requests for quarterly indexes) form4_filings = get_filings(form="4", year=2024)` ### 5\. Implement Exponential Backoff For custom requests outside of edgartools: `import time import random def request_with_backoff(url, max_retries=5): retries = 0 while retries < max_retries: try: # Make request response = make_request(url) return response except Exception as e: if "429" in str(e) or "403" in str(e): # Calculate backoff time wait_time = (2 ** retries) + random.random() print(f"Rate limited. Waiting {wait_time:.1f} seconds...") time.sleep(wait_time) retries += 1 else: raise raise Exception("Max retries exceeded")` Handling Rate Limit Errors -------------------------- If you encounter rate limit errors despite following best practices: 1. **Reduce your request rate** by setting a lower rate limit 2. **Increase delays** between requests 3. **Implement circuit breakers** to pause requests when errors occur 4. **Spread requests** across a longer time period 5. **Use a different network** if your IP has been temporarily blocked SEC Access Hours ---------------- While the SEC EDGAR system is available 24/7, it's good practice to avoid peak hours: * **Peak hours**: 9:30 AM - 4:00 PM Eastern Time (market hours) * **Maintenance**: Occasionally on weekends For large batch operations, consider running them during off-peak hours. Additional Compliance Considerations ------------------------------------ ### Terms of Service The SEC provides EDGAR data as a public service. When using this data: * Don't misrepresent the data or its source * Don't claim affiliation with the SEC * Provide proper attribution when republishing data ### Privacy Considerations Some SEC filings contain personal information. Be mindful of privacy concerns when: * Storing filings locally * Processing personal information in filings * Republishing or sharing filing data Monitoring Your Usage --------------------- To monitor your usage and ensure compliance: `from edgar import get_request_stats # Get statistics about your requests stats = get_request_stats() print(f"Requests made: {stats['total_requests']}") print(f"Average rate: {stats['average_rate_per_second']:.2f} requests/second") print(f"Rate limit errors: {stats['rate_limit_errors']}")` Conclusion ---------- Complying with SEC EDGAR access requirements is straightforward with edgartools. By setting your identity, respecting rate limits, and following best practices, you can ensure reliable and compliant access to SEC filing data. Remember that the SEC provides this valuable data as a public service. Responsible usage helps ensure that EDGAR remains accessible to everyone. Additional Resources -------------------- * [SEC EDGAR Fair Access Policy](https://www.sec.gov/os/accessing-edgar-data) * [SEC Developer Resources](https://www.sec.gov/developer) * [SEC EDGAR Robots.txt](https://www.sec.gov/robots.txt) Back to top --- # Proxy Statements (DEF 14A) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/proxystatement-data-object-guide/#proxy-statements-def-14a-parse-executive-compensation-and-governance-data) Proxy Statements (DEF 14A): Parse Executive Compensation and Governance Data ============================================================================ Form DEF 14A is a definitive proxy statement filed by public companies before annual shareholder meetings. It contains critical information about executive compensation, board composition, shareholder voting matters, and corporate governance. This guide details all data available from the `ProxyStatement` class for building views. * * * Overview -------- | Property | Type | Description | | --- | --- | --- | | Class Name | `ProxyStatement` | | | Forms Handled | `DEF 14A`, `DEFA14A`, `DEFM14A`, `DEF 14A/A` | | | Module | `edgar.proxy` | | | Source Data | XBRL (primary) + HTML (secondary) | | ### Form Type Descriptions | Form | Description | | --- | --- | | `DEF 14A` | Definitive Proxy Statement - standard proxy filing | | `DEFA14A` | Definitive Additional Proxy Soliciting Materials | | `DEFM14A` | Definitive Proxy Statement relating to Merger or Acquisition | | `DEF 14A/A` | Amendment to Definitive Proxy Statement | ### Data Source Reliability | Source | Reliability | Description | | --- | --- | --- | | XBRL | High | Executive compensation, pay vs performance - standardized across all companies | | HTML | Medium | Beneficial ownership, board info, proposals - requires parsing | * * * Basic Metadata -------------- | Property | Type | Description | Example | | --- | --- | --- | --- | | `form` | `str` | Form type | `"DEF 14A"` | | `filing_date` | `str` | Date filed with SEC | `"2025-01-10"` | | `fiscal_year_end` | `str` | Fiscal year end date | `"2024-09-28"` | | `company_name` | `str` | Company legal name | `"Apple Inc."` | | `cik` | `str` | Central Index Key | `"0000320193"` | | `accession_number` | `str` | SEC accession number | `"0001308179-25-000008"` | * * * Executive Compensation (XBRL - High Reliability) ------------------------------------------------ Executive compensation data is extracted from XBRL using the SEC's Executive Compensation Disclosure (ECD) taxonomy. This data is highly standardized and available for all companies. ### PEO (Principal Executive Officer / CEO) | Property | Type | XBRL Concept | Description | Example | | --- | --- | --- | --- | --- | | `peo_name` | `str` | `ecd:PeoName` | CEO name | `"Mr. Cook"` | | `peo_total_comp` | `Decimal` | `ecd:PeoTotalCompAmt` | Total compensation from Summary Compensation Table | `74,609,802` | | `peo_actually_paid_comp` | `Decimal` | `ecd:PeoActuallyPaidCompAmt` | Compensation Actually Paid (CAP) | `168,980,568` | ### Non-PEO Named Executive Officers (NEOs) | Property | Type | XBRL Concept | Description | Example | | --- | --- | --- | --- | --- | | `neo_avg_total_comp` | `Decimal` | `ecd:NonPeoNeoAvgTotalCompAmt` | Average NEO total compensation | `27,178,896` | | `neo_avg_actually_paid_comp` | `Decimal` | `ecd:NonPeoNeoAvgCompActuallyPaidAmt` | Average NEO compensation actually paid | `58,633,525` | ### Compensation Time Series (5 Years) The `executive_compensation` property returns a DataFrame with 5 years of compensation data: `proxy = filing.obj() comp_df = proxy.executive_compensation # pd.DataFrame` | Column | Type | Description | | --- | --- | --- | | `fiscal_year_end` | `date` | End of fiscal year | | `peo_total_comp` | `Decimal` | PEO total from SCT | | `peo_actually_paid_comp` | `Decimal` | PEO compensation actually paid | | `neo_avg_total_comp` | `Decimal` | Non-PEO NEO average total | | `neo_avg_actually_paid_comp` | `Decimal` | Non-PEO NEO average CAP | ### Example Output `# Apple Inc. Executive Compensation (5 years) fiscal_year_end peo_total_comp peo_actually_paid_comp neo_avg_total_comp neo_avg_actually_paid 2024-09-28 74,609,802 168,980,568 27,178,896 58,633,525 2023-09-30 63,209,845 106,643,588 26,938,240 48,892,163 2022-09-24 99,420,097 128,833,021 26,929,095 35,842,114 2021-09-25 98,734,394 311,845,801 26,989,456 89,764,231 2020-09-26 14,769,259 4,567,123 23,976,158 12,589,743` ### Named Executives (Dimensional Data) Some companies tag individual executive data using dimensional XBRL. When available: `# Check if individual executive data is available if proxy.has_individual_executive_data: executives = proxy.named_executives # list of executive dicts for exec in executives: print(f"{exec['name']}: ${exec['actually_paid_comp']:,}")` | Property | Type | Description | | --- | --- | --- | | `has_individual_executive_data` | `bool` | Whether individual executive dimensions are available | | `named_executives` | `list[dict]` | Individual executive compensation details (when available) | **Note**: Only ~60% of companies use dimensional tagging (AAPL, JPM, JNJ). Others aggregate to PEO vs Non-PEO NEO averages (MSFT, XOM). * * * Pay vs Performance (XBRL - High Reliability) -------------------------------------------- Pay vs Performance disclosures correlate executive compensation with company performance metrics. ### Primary Metrics | Property | Type | XBRL Concept | Description | Example | | --- | --- | --- | --- | --- | | `total_shareholder_return` | `Decimal` | `ecd:TotalShareholderRtnAmt` | Company TSR (cumulative %) | `207.6` | | `peer_group_tsr` | `Decimal` | `ecd:PeerGroupTotalShareholderRtnAmt` | Peer group TSR | `189.3` | | `net_income` | `Decimal` | `us-gaap:NetIncomeLoss` | Net income (USD) | `93,736,000,000` | ### Company-Selected Performance Measure | Property | Type | XBRL Concept | Description | Example | | --- | --- | --- | --- | --- | | `company_selected_measure` | `str` | `ecd:CoSelectedMeasureName` | Company's chosen KPI name | `"Operating Cash Flow"` | | `company_selected_measure_value` | `Decimal` | `ecd:CoSelectedMeasureAmt` | KPI value | `118,254,000,000` | ### Most Important Performance Measures | Property | Type | XBRL Concept | Description | | --- | --- | --- | --- | | `performance_measures` | `list[str]` | `ecd:MeasureName` | List of performance measures used | Example values: `["Revenue", "Operating Income", "Free Cash Flow", "Total Shareholder Return"]` ### Pay vs Performance DataFrame `pvp_df = proxy.pay_vs_performance # pd.DataFrame` | Column | Type | Description | | --- | --- | --- | | `fiscal_year_end` | `date` | End of fiscal year | | `peo_actually_paid_comp` | `Decimal` | CEO compensation actually paid | | `neo_avg_actually_paid_comp` | `Decimal` | NEO average CAP | | `total_shareholder_return` | `Decimal` | Company TSR | | `peer_group_tsr` | `Decimal` | Peer group TSR | | `net_income` | `Decimal` | Net income | | `company_selected_measure_value` | `Decimal` | Company KPI value | * * * Governance Indicators (XBRL) ---------------------------- | Property | Type | XBRL Concept | Description | Example | | --- | --- | --- | --- | --- | | `insider_trading_policy_adopted` | `bool` | `ecd:InsiderTrdPoliciesProcAdoptedFlag` | Has adopted insider trading policy | `True` | * * * XBRL Concept Reference ---------------------- ### Universal Concepts (Present in ALL Companies) These 25 concepts are available across all sampled DEF 14A filings (100% coverage): #### Executive Compensation | Concept | Description | | --- | --- | | `ecd:PeoTotalCompAmt` | PEO total compensation from Summary Compensation Table | | `ecd:PeoActuallyPaidCompAmt` | PEO compensation actually paid | | `ecd:NonPeoNeoAvgTotalCompAmt` | Non-PEO NEO average total compensation | | `ecd:NonPeoNeoAvgCompActuallyPaidAmt` | Non-PEO NEO average compensation actually paid | | `ecd:AdjToCompAmt` | Adjustments to compensation (reconciliation) | | `ecd:PeoName` | Name of Principal Executive Officer | #### Performance Metrics | Concept | Description | | --- | --- | | `ecd:TotalShareholderRtnAmt` | Company total shareholder return | | `ecd:PeerGroupTotalShareholderRtnAmt` | Peer group total shareholder return | | `us-gaap:NetIncomeLoss` | Net income (GAAP) | | `ecd:CoSelectedMeasureAmt` | Company-selected performance measure value | | `ecd:CoSelectedMeasureName` | Company-selected performance measure name | | `ecd:MeasureName` | Names of most important performance measures | #### Text Blocks and Footnotes | Concept | Description | | --- | --- | | `ecd:PvpTableTextBlock` | Pay vs Performance table text block | | `ecd:TabularListTableTextBlock` | Tabular list of performance measures | | `ecd:NamedExecutiveOfficersFnTextBlock` | Named executives footnote | | `ecd:PeerGroupIssuersFnTextBlock` | Peer group issuers footnote | | `ecd:AdjToPeoCompFnTextBlock` | PEO compensation adjustment footnote | | `ecd:AdjToNonPeoNeoCompFnTextBlock` | Non-PEO NEO adjustment footnote | | `ecd:CompActuallyPaidVsTotalShareholderRtnTextBlock` | CAP vs TSR discussion | | `ecd:CompActuallyPaidVsNetIncomeTextBlock` | CAP vs Net Income discussion | | `ecd:CompActuallyPaidVsCoSelectedMeasureTextBlock` | CAP vs company measure discussion | #### Governance | Concept | Description | | --- | --- | | `ecd:InsiderTrdPoliciesProcAdoptedFlag` | Insider trading policy adoption flag | * * * Code Examples ------------- ### Example 1: Extract Executive Compensation `from edgar import Company # Get company and filing company = Company("AAPL") filing = company.get_filings(form="DEF 14A").latest() # Get proxy statement object proxy = filing.obj() # Access executive compensation print(f"CEO: {proxy.peo_name}") print(f"CEO Total Compensation: ${proxy.peo_total_comp:,}") print(f"CEO Compensation Actually Paid: ${proxy.peo_actually_paid_comp:,}") print(f"NEO Average Compensation: ${proxy.neo_avg_actually_paid_comp:,}")` ### Example 2: Pay vs Performance Analysis `from edgar import Company company = Company("MSFT") filing = company.get_filings(form="DEF 14A").latest() proxy = filing.obj() # Get pay vs performance DataFrame pvp = proxy.pay_vs_performance # Calculate pay-for-performance correlation correlation = pvp['peo_actually_paid_comp'].corr(pvp['total_shareholder_return']) print(f"CEO Pay vs TSR Correlation: {correlation:.2f}") # Compare to peer group print(f"Company TSR: {proxy.total_shareholder_return}%") print(f"Peer Group TSR: {proxy.peer_group_tsr}%") print(f"Outperformance: {proxy.total_shareholder_return - proxy.peer_group_tsr:.1f}%")` ### Example 3: Governance Check `from edgar import Company company = Company("JPM") filing = company.get_filings(form="DEF 14A").latest() proxy = filing.obj() # Check governance indicators if proxy.insider_trading_policy_adopted: print("Insider Trading Policy: Adopted") else: print("Insider Trading Policy: Not Adopted (flag for review)") # List performance measures used print("Performance Measures:") for measure in proxy.performance_measures: print(f" - {measure}")` ### Example 4: Multi-Company Comparison `from edgar import Company import pandas as pd tickers = ["AAPL", "MSFT", "GOOGL", "META", "AMZN"] data = [] for ticker in tickers: company = Company(ticker) filing = company.get_filings(form="DEF 14A").latest() proxy = filing.obj() data.append({ 'company': company.name, 'ceo': proxy.peo_name, 'ceo_total_comp': proxy.peo_total_comp, 'ceo_actually_paid': proxy.peo_actually_paid_comp, 'tsr': proxy.total_shareholder_return, 'peer_tsr': proxy.peer_group_tsr }) comparison_df = pd.DataFrame(data) print(comparison_df.to_string())` ### Example 5: Access Board and Director Information The `ProxyStatement` class focuses on XBRL-based executive compensation data. Board composition, director details, and shareholder proposals live in the HTML body of the filing and are not yet extracted into structured properties. However, you can access this information today using the `Filing` object's built-in search and HTML capabilities. `from edgar import Company # Get a DEF 14A filing company = Company("AAPL") filing = company.get_filings(form="DEF 14A").latest() # Search the filing HTML for board-related sections results = filing.search("board of directors") for section in results[:3]: print(section[:200]) # Preview matching sections` `# Search for specific governance topics director_sections = filing.search("director nominees") ownership_sections = filing.search("beneficial ownership") proposal_sections = filing.search("proposal") audit_sections = filing.search("audit fees")` `# Get the full HTML for manual inspection or custom parsing html_content = filing.html() # Or access the filing document directly doc = filing.document()` > **Note**: Board composition, director bios, beneficial ownership tables, and shareholder proposals are available in the filing HTML but require custom parsing. Structured `Director` and `Proposal` objects are planned for a future release. See the [HTML-Based Data](https://edgartools.readthedocs.io/en/stable/guides/proxystatement-data-object-guide/#html-based-data-future-features) > section below for details on what data is available and extraction patterns. * * * View Design Recommendations --------------------------- ### Primary View Components 1. **Header Section** 2. Company name (prominent) 3. Form type with amendment indicator 4. Filing date and fiscal year end 5. Annual meeting date (if available) 6. **Compensation Dashboard** 7. CEO compensation card (Total SCT vs Actually Paid) 8. NEO average compensation card 9. 5-year trend sparkline or chart 10. Year-over-year change indicators 11. **Pay vs Performance Panel** 12. TSR comparison chart (company vs peer group) 13. Compensation vs TSR correlation visualization 14. Net income trend overlay 15. Company-selected performance measure 16. **Governance Indicators** 17. Insider trading policy status badge 18. Performance measures list 19. **Key Metrics Cards** 20. Total Shareholder Return 21. Peer Group TSR 22. Net Income 23. Company KPI with label ### Data Priority for Display | Priority | Data | Reason | | --- | --- | --- | | High | CEO compensation (both SCT and CAP) | Primary user interest | | High | Total Shareholder Return | Key performance metric | | High | Peer group comparison | Benchmark context | | Medium | NEO average compensation | Executive team context | | Medium | Net income | Financial performance | | Medium | Company-selected measure | Company's chosen KPI | | Medium | 5-year compensation trends | Historical context | | Low | Adjustment details | Technical reconciliation | | Low | Footnote text blocks | Reference material | ### Value Formatting | Data Type | Format | Example | | --- | --- | --- | | Compensation | Currency with commas | `$168,980,568` | | Large values (>$1B) | Abbreviated | `$93.7B` | | TSR | Percentage with 1 decimal | `207.6%` | | Year-over-year change | Signed percentage | `+12.5%` or `-8.3%` | ### Visual Indicators (Suggested) | Condition | Visual Treatment | | --- | --- | | Amendment (`/A`) | Yellow "Amendment" badge | | TSR > Peer TSR | Green upward arrow | | TSR < Peer TSR | Red downward arrow | | Insider policy adopted | Green checkmark | | Insider policy not adopted | Red warning icon | | Compensation increase >25% YoY | Orange highlight | | Compensation decrease | Blue highlight | ### Compensation Card Layout `+----------------------------------+ | CEO Compensation | | Mr. Tim Cook | +----------------------------------+ | Summary Comp Table | | $74,609,802 | | +18.0% vs prior year | +----------------------------------+ | Compensation Actually Paid | | $168,980,568 | | +58.4% vs prior year | +----------------------------------+` * * * Example Data Structure ---------------------- `{ # Metadata "form": "DEF 14A", "filing_date": "2025-01-10", "fiscal_year_end": "2024-09-28", "company_name": "Apple Inc.", "cik": "0000320193", "accession_number": "0001308179-25-000008", # Executive Compensation "peo_name": "Mr. Cook", "peo_total_comp": 74609802, "peo_actually_paid_comp": 168980568, "neo_avg_total_comp": 27178896, "neo_avg_actually_paid_comp": 58633525, # Pay vs Performance "total_shareholder_return": 207.6, "peer_group_tsr": 189.3, "net_income": 93736000000, "company_selected_measure": "Operating Cash Flow", "company_selected_measure_value": 118254000000, # Performance Measures "performance_measures": [ "Net Sales", "Operating Income", "Total Shareholder Return", "Operating Cash Flow" ], # Governance "insider_trading_policy_adopted": True, # Named Executives (when dimensional data available) "has_individual_executive_data": True, "named_executives": [ { "id": "aapl:CookMember", "name": "Mr. Cook", "role": "PEO", "total_comp": 74609802, "actually_paid_comp": 168980568 }, { "id": "aapl:MaestriMember", "name": "Luca Maestri", "role": "NEO", "total_comp": 27178896, "actually_paid_comp": 58633525 } ], # Time Series (5 years) "executive_compensation": [ { "fiscal_year_end": "2024-09-28", "peo_total_comp": 74609802, "peo_actually_paid_comp": 168980568, "neo_avg_total_comp": 27178896, "neo_avg_actually_paid_comp": 58633525 }, { "fiscal_year_end": "2023-09-30", "peo_total_comp": 63209845, "peo_actually_paid_comp": 106643588, "neo_avg_total_comp": 26938240, "neo_avg_actually_paid_comp": 48892163 } # ... 3 more years ], # Pay vs Performance Series "pay_vs_performance": [ { "fiscal_year_end": "2024-09-28", "peo_actually_paid_comp": 168980568, "neo_avg_actually_paid_comp": 58633525, "total_shareholder_return": 207.6, "peer_group_tsr": 189.3, "net_income": 93736000000 } # ... more years ] }` * * * HTML-Based Data (Future Features) --------------------------------- The following data is available in DEF 14A HTML sections but not yet extracted into structured properties. You can access these sections today using `filing.search()` to find relevant content. ### Beneficial Ownership | Section | Description | | --- | --- | | Principal Shareholders | Shareholders owning >5% of shares | | Director/Executive Ownership | Shares owned by insiders | **How to access today**: `results = filing.search("beneficial ownership") # or results = filing.search("security ownership")` ### Board of Directors | Data | Description | | --- | --- | | Director Names | Full list of board members | | Director Ages | Ages of directors | | Director Tenure | Years on board | | Independence Status | Independent vs non-independent | | Committee Memberships | Audit, Compensation, Governance | **How to access today**: `results = filing.search("board of directors") # or results = filing.search("director nominees")` ### Director Compensation | Data | Description | | --- | --- | | Director Fees | Annual retainer and meeting fees | | Stock Awards | Equity compensation | | Total Compensation | Sum of all compensation | **How to access today**: `results = filing.search("director compensation")` ### Voting Proposals | Proposal Type | Description | | --- | --- | | Election of Directors | Board member elections | | Ratification of Auditors | Audit firm approval | | Say-on-Pay | Executive compensation advisory vote | | Shareholder Proposals | Proposals submitted by shareholders | | Equity Plan Amendments | Stock compensation plan changes | **How to access today**: `results = filing.search("proposal")` ### Audit Information | Data | Description | | --- | --- | | Auditor Name | Independent auditor firm | | Audit Fees | Fees for audit services | | Tax Fees | Fees for tax services | | Other Fees | Other professional fees | **How to access today**: `results = filing.search("audit fees")` * * * Notes for Implementation ------------------------ 1. **XBRL Namespace**: The primary namespace for proxy data is `ecd:` (Executive Compensation Disclosure), introduced by SEC in 2022 for fiscal years ending on or after December 16, 2022. 2. **Dimensional Tagging Variation**: 3. ~60% of companies tag individual executive data using `dim_ecd_IndividualAxis` 4. ~40% only provide aggregate data (PEO and Non-PEO NEO averages) 5. Always check `has_individual_executive_data` before accessing individual executives 6. **Time Series Data**: Pay vs Performance tables include 5 years of historical data per SEC requirements. This enables robust trend analysis. 7. **Compensation Actually Paid (CAP)**: This SEC-mandated metric differs from Summary Compensation Table totals due to adjustments for: 8. Change in pension value 9. Stock/option awards fair value changes 10. Vesting date fair values 11. **Company-Selected Measure**: Each company chooses one performance measure they consider most important. Common choices include: 12. Revenue or Net Sales 13. Operating Income 14. Free Cash Flow 15. Adjusted EBITDA 16. Return on Invested Capital 17. **Peer Group**: Companies define their own peer groups for TSR comparison. The peer group composition is disclosed in the filing but not structured in XBRL. 18. **Amendment Handling**: DEF 14A/A filings contain corrections or updates. Always use the most recent filing for a given fiscal year. 19. **Form Variants**: 20. `DEF 14A`: Standard proxy statement 21. `DEFA14A`: Additional soliciting materials (may have limited data) 22. `DEFM14A`: Merger-related proxy (may have different structure) 23. **Fiscal Year Alignment**: Match compensation periods with company fiscal year ends, which vary by company (e.g., Apple uses September, Microsoft uses June). 24. **Large Cap Coverage**: XBRL data is highly reliable for S&P 500 companies. Smaller companies may have less complete tagging. Back to top --- # SEC Rate Limits & Compliance - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/resources/sec-compliance/#sec-rate-limits-compliance) SEC Rate Limits & Compliance ============================ The SEC EDGAR system is a valuable public resource that provides access to corporate filings. To ensure fair access for all users, the SEC has established guidelines and rate limits for automated access. This guide explains these requirements and how to use edgartools in a compliant manner. SEC EDGAR Access Requirements ----------------------------- ### Fair Access Policy The SEC maintains a [Fair Access Policy](https://www.sec.gov/os/accessing-edgar-data) that requires all automated EDGAR access to: 1. Identify the accessing user/organization in the HTTP request 2. Limit request rates to avoid overloading the system 3. Respect the `robots.txt` directives 4. Access data during appropriate hours ### Required Identity Information When using automated tools to access EDGAR, you must identify yourself by providing: * Your name or organization name * Your email address This allows the SEC to contact you if there are issues with your access patterns. Setting Your Identity in edgartools ----------------------------------- edgartools makes it easy to comply with SEC requirements by providing a simple way to set your identity: `from edgar import set_identity # Set your identity information set_identity( name="Your Name", email="your.email@example.com", organization="Your Organization" # Optional )` This identity information will be included in the `User-Agent` header of all requests made by edgartools. ### Default Behavior If you don't explicitly set your identity, edgartools will: 1. Look for environment variables `EDGAR_NAME` and `EDGAR_EMAIL` 2. If not found, use a generic identity that indicates edgartools usage However, it's strongly recommended to set your own identity to ensure compliance with SEC requirements. Understanding SEC Rate Limits ----------------------------- The SEC doesn't publish specific rate limits, but based on their guidelines and observed behavior, the following limits are recommended: * No more than 10 requests per second * Reasonable total volume per day * Avoid excessive concurrent requests ### edgartools Default Rate Limiting By default, edgartools implements conservative rate limiting: * Maximum of 10 requests per second * Built-in delays between requests * Automatic retries with exponential backoff for 429 errors This default configuration is designed to keep you compliant with SEC guidelines while still providing good performance. Customizing Rate Limits ----------------------- You can adjust the rate limits in edgartools if needed: `from edgar import set_rate_limit # Set a more conservative rate limit (requests per second) set_rate_limit(5) # 5 requests per second` For high-volume or production use cases, consider being more conservative with your rate limits to avoid potential IP blocks. Signs of Exceeding Rate Limits ------------------------------ If you exceed SEC rate limits, you may experience: 1. HTTP 429 (Too Many Requests) responses 2. HTTP 403 (Forbidden) responses 3. Temporary IP blocks (typically 10 minutes to 24 hours) edgartools will automatically handle 429 responses with retries, but persistent rate limit violations may result in longer blocks. Best Practices for Compliant Access ----------------------------------- ### 1\. Always Set Your Identity `from edgar import set_identity set_identity( name="Your Name", email="your.email@example.com" )` ### 2\. Use Local Storage Reduce the number of requests by storing filings locally: `from edgar import enable_local_storage enable_local_storage("/path/to/storage")` ### 3\. Implement Appropriate Delays For batch processing, add delays between operations: `import time for filing in filings: # Process filing process_filing(filing) # Add delay between filings time.sleep(0.2) # 200ms delay` ### 4\. Use Efficient Query Patterns Choose the most efficient access pattern for your needs: `# For company-specific queries, use company.get_filings() # (makes just one request for all filings) company = Company("AAPL") filings = company.get_filings(form="10-K") # For form-specific queries across companies, use get_filings() # (makes requests for quarterly indexes) form4_filings = get_filings(form="4", year=2024)` ### 5\. Implement Exponential Backoff For custom requests outside of edgartools: `import time import random def request_with_backoff(url, max_retries=5): retries = 0 while retries < max_retries: try: # Make request response = make_request(url) return response except Exception as e: if "429" in str(e) or "403" in str(e): # Calculate backoff time wait_time = (2 ** retries) + random.random() print(f"Rate limited. Waiting {wait_time:.1f} seconds...") time.sleep(wait_time) retries += 1 else: raise raise Exception("Max retries exceeded")` Handling Rate Limit Errors -------------------------- If you encounter rate limit errors despite following best practices: 1. **Reduce your request rate** by setting a lower rate limit 2. **Increase delays** between requests 3. **Implement circuit breakers** to pause requests when errors occur 4. **Spread requests** across a longer time period 5. **Use a different network** if your IP has been temporarily blocked SEC Access Hours ---------------- While the SEC EDGAR system is available 24/7, it's good practice to avoid peak hours: * **Peak hours**: 9:30 AM - 4:00 PM Eastern Time (market hours) * **Maintenance**: Occasionally on weekends For large batch operations, consider running them during off-peak hours. Additional Compliance Considerations ------------------------------------ ### Terms of Service The SEC provides EDGAR data as a public service. When using this data: * Don't misrepresent the data or its source * Don't claim affiliation with the SEC * Provide proper attribution when republishing data ### Privacy Considerations Some SEC filings contain personal information. Be mindful of privacy concerns when: * Storing filings locally * Processing personal information in filings * Republishing or sharing filing data Monitoring Your Usage --------------------- To monitor your usage and ensure compliance: `from edgar import get_request_stats # Get statistics about your requests stats = get_request_stats() print(f"Requests made: {stats['total_requests']}") print(f"Average rate: {stats['average_rate_per_second']:.2f} requests/second") print(f"Rate limit errors: {stats['rate_limit_errors']}")` Conclusion ---------- Complying with SEC EDGAR access requirements is straightforward with edgartools. By setting your identity, respecting rate limits, and following best practices, you can ensure reliable and compliant access to SEC filing data. Remember that the SEC provides this valuable data as a public service. Responsible usage helps ensure that EDGAR remains accessible to everyone. Additional Resources -------------------- * [SEC EDGAR Fair Access Policy](https://www.sec.gov/os/accessing-edgar-data) * [SEC Developer Resources](https://www.sec.gov/developer) * [SEC EDGAR Robots.txt](https://www.sec.gov/robots.txt) Back to top --- # Filing - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/api/filing/#filing-api-reference-access-sec-filing-content-xbrl-data-and-documents) Filing API Reference — Access SEC Filing Content, XBRL Data, and Documents ========================================================================== The `Filing` class represents a single SEC filing and provides comprehensive access to its content, structured data, attachments, and metadata. **Quick example:** `from edgar import get_by_accession_number filing = get_by_accession_number("0000320193-23-000106") print(f"{filing.company} {filing.form} filed {filing.filing_date}") # Access content text = filing.text() xbrl = filing.xbrl() # Get form-specific object tenk = filing.obj() # Returns TenK for 10-K filings` Getting a Filing ---------------- `from edgar import Company, get_filings, get_by_accession_number # From a company company = Company("AAPL") filing = company.get_filings(form="10-K").latest() # From a search filings = get_filings(2024, 1, form="10-K") filing = filings[0] # By accession number filing = get_by_accession_number("0000320193-23-000106")` Core Properties --------------- ### Basic Information | Property | Type | Description | | --- | --- | --- | | `cik` | int | Central Index Key of the filing entity | | `company` | str | Company name | | `form` | str | SEC form type (e.g., "10-K", "10-Q", "8-K") | | `filing_date` | str | Date filed with SEC (YYYY-MM-DD) | | `accession_no` | str | Unique SEC accession number | | `accession_number` | str | Alias for `accession_no` | | `period_of_report` | str | Reporting period end date | **Example:** `print(filing.cik) # 320193 print(filing.company) # "Apple Inc." print(filing.form) # "10-K" print(filing.filing_date) # "2023-11-03" print(filing.period_of_report) # "2023-09-30"` ### Document Properties | Property | Type | Description | | --- | --- | --- | | `document` | Attachment | Primary display document | | `primary_documents` | List\[Attachment\] | All primary documents | | `attachments` | Attachments | All documents and attachments | | `exhibits` | Attachments | Exhibits only (subset of attachments) | **Example:** `# Access primary document doc = filing.document print(doc.document_type) # Loop through all attachments for att in filing.attachments: print(f"{att.sequence}: {att.description}") # Access exhibits for exhibit in filing.exhibits: print(f"Exhibit {exhibit.exhibit_number}: {exhibit.description}")` ### URL Properties | Property | Description | | --- | --- | | `homepage_url` | Filing homepage on SEC website | | `filing_url` | URL to primary filing document | | `text_url` | URL to text version | | `base_dir` | Base directory for all filing files | **Example:** `print(filing.homepage_url) # https://www.sec.gov/Archives/edgar/data/320193/000032019323000106/0000320193-23-000106-index.html print(filing.filing_url) # https://www.sec.gov/Archives/edgar/data/320193/000032019323000106/aapl-20230930.htm` ### Metadata Properties | Property | Type | Description | | --- | --- | --- | | `header` | FilingHeader | Parsed SGML header information | | `is_multi_entity` | bool | Whether filing involves multiple entities | | `all_ciks` | List\[int\] | All CIK numbers in filing | | `all_entities` | List\[str\] | All entity names in filing | Content Access Methods ---------------------- ### Raw Content #### html() `def html(self) -> Optional[str]` Get HTML content of the primary document. **Returns:** HTML string or None if not available **Example:** `html = filing.html() if html: print(f"HTML length: {len(html)} characters")` #### text() `def text(self) -> str` Convert filing HTML to clean plain text. **Returns:** Plain text content **Example:** `text = filing.text() # Search within text if "artificial intelligence" in text.lower(): print("AI mentioned in filing")` #### markdown() `def markdown( include_page_breaks: bool = False, start_page_number: int = 0 ) -> str` Convert filing to Markdown format. **Parameters:** - `include_page_breaks` (bool): Include page break markers - `start_page_number` (int): Starting page number for page breaks **Returns:** Markdown formatted content **Example:** `md = filing.markdown() with open("filing.md", "w") as f: f.write(md)` #### xml() `def xml(self) -> Optional[str]` Get XML content if filing contains XML data. **Returns:** XML string or None **Example:** `xml = filing.xml() if xml: import xml.etree.ElementTree as ET root = ET.fromstring(xml)` #### full\_text\_submission() `def full_text_submission(self) -> str` Get the complete SEC text submission file. **Returns:** Full submission text including SGML headers ### Structured Data Access #### xbrl() `def xbrl(self) -> Optional[XBRL]` Get XBRL data object if filing contains XBRL. **Returns:** `XBRL` object or None **Example:** `xbrl = filing.xbrl() if xbrl: # Access financial statements income = xbrl.statements.income_statement() balance = xbrl.statements.balance_sheet() cashflow = xbrl.statements.cash_flow_statement()` **See also:** [XBRL API Reference](https://edgartools.readthedocs.io/en/latest/api/xbrl/) , [Extract Financial Statements Guide](https://edgartools.readthedocs.io/en/latest/guides/extract-statements/) #### obj() / data\_object() `def obj(self) def data_object(self) # Alias` Get form-specific structured object based on filing type. **Returns:** Form-specific object (TenK, TenQ, EightK, Form4, etc.) **Form type mappings:** | Form Type | Return Class | Module | | --- | --- | --- | | 10-K | TenK | edgar.company\_reports | | 10-Q | TenQ | edgar.company\_reports | | 8-K | EightK | edgar.company\_reports | | 20-F | TwentyF | edgar.company\_reports | | 4 | Form4 | edgar.ownership | | 3 | Form3 | edgar.ownership | | 5 | Form5 | edgar.ownership | | DEF 14A | ProxyStatement | edgar.proxy | | 13F-HR | ThirteenF | edgar.holdings | | SC 13D/G | Schedule13 | edgar.ownership | | NPORT-P | NportFiling | edgar.nport | | 144 | Form144 | edgar.ownership | **Example:** `# For a 10-K filing tenk = filing.obj() print(type(tenk)) # # Access financial statements from TenK object if tenk.financials: income = tenk.financials.income_statement balance = tenk.financials.balance_sheet cashflow = tenk.financials.cash_flow_statement # Or use direct properties income = tenk.income_statement balance = tenk.balance_sheet # XBRL report pages (also available via filing.reports) reports = tenk.reports` **Important:** The base `Filing` class does **not** have a `financials` property. To access financial data: - Use `filing.obj().financials` for 10-K/10-Q filings - Or use `filing.xbrl().statements` for any XBRL filing **Incorrect:** `# This will fail - Filing has no financials property financials = filing.financials # AttributeError` **Correct:** `# Get form-specific object first tenk = filing.obj() if tenk.financials: financials = tenk.financials # Or use XBRL directly xbrl = filing.xbrl() if xbrl: statements = xbrl.statements` ### Parsing and Search #### parse() `def parse(self) -> Document` Parse filing into structured Document for advanced searching. **Returns:** Parsed `Document` object **Example:** `doc = filing.parse() # Use document methods for structured search` #### search() `def search(self, query: str, regex: bool = False) -> List[str]` Search for text within filing content. **Parameters:** - `query` (str): Search term or pattern - `regex` (bool): Treat query as regex pattern **Returns:** List of matching text excerpts **Example:** `# Simple text search results = filing.search("revenue recognition") print(f"Found {len(results)} mentions") # Regex search for emails emails = filing.search(r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b', regex=True)` #### sections() `def sections(self) -> List[str]` Get list of available document sections. **Returns:** List of section names/identifiers **Example:** `sections = filing.sections() for section in sections: print(section) # "Item 1", "Item 2", etc.` #### sgml() `def sgml(self) -> FilingSGML` Get parsed SGML structure of filing. **Returns:** `FilingSGML` object with document structure **Example:** `sgml = filing.sgml() for doc in sgml.documents: print(f"{doc.type}: {doc.sequence}")` Interactive Methods ------------------- ### Viewing and Display #### view() `def view(self)` Display filing in console or Jupyter notebook with Rich formatting. **Example:** `filing.view() # Shows formatted filing content` #### open() `def open(self)` Open primary filing document in default web browser. **Example:** `filing.open() # Opens filing in browser` #### open\_homepage() `def open_homepage(self)` Open filing homepage (index page) in default web browser. **Example:** `filing.open_homepage() # Opens SEC filing index page` #### serve() `def serve(self, port: int = 8000)` Serve filing on local HTTP server for viewing. **Parameters:** - `port` (int): Server port (default: 8000) **Example:** `filing.serve(port=8080) # Access at http://localhost:8080` Entity and Related Data ----------------------- ### get\_entity() `def get_entity(self) -> Union[Company, Entity]` Get Company or Entity object for this filing. **Returns:** `Company` or `Entity` instance **Example:** `entity = filing.get_entity() print(f"Entity: {entity.name}") print(f"Industry: {entity.industry}")` ### as\_company\_filing() `def as_company_filing(self) -> EntityFiling` Convert to EntityFiling with enhanced metadata. **Returns:** `EntityFiling` object ### related\_filings() `def related_filings(self) -> Filings` Get filings related by file number. **Returns:** `Filings` collection Persistence and Serialization ----------------------------- ### Save and Load #### save() `def save(self, directory_or_file: PathLike)` Save filing using pickle serialization. **Parameters:** - `directory_or_file`: Directory or file path **Example:** `# Save to directory filing.save("./data/filings/") # Save to specific file filing.save("./data/apple_10k_2023.pkl")` #### load() `@classmethod def load(cls, path: PathLike) -> Filing` Load filing from pickle file. **Parameters:** - `path`: Path to pickle file **Returns:** `Filing` object **Example:** `filing = Filing.load("./data/apple_10k_2023.pkl")` ### Data Export #### to\_dict() `def to_dict(self) -> Dict[str, Union[str, int]]` Convert to dictionary representation. **Returns:** Dictionary with filing data **Example:** `data = filing.to_dict() print(data.keys()) # dict_keys(['cik', 'company', 'form', 'filing_date', 'accession_no', ...])` #### from\_dict() `@classmethod def from_dict(cls, data: Dict) -> Filing` Create Filing from dictionary. **Parameters:** - `data`: Dictionary with filing information **Returns:** `Filing` object #### summary() `def summary(self) -> pd.DataFrame` Get filing summary as pandas DataFrame. **Returns:** DataFrame with filing metadata #### to\_context() `def to_context(self, detail: str) -> str` Generate context string for LLM/AI use. **Parameters:** - `detail` (str): Level of detail **Returns:** Context string ### Download #### download() `def download( self, data_directory: Optional[str] = None, compress: bool = True, compression_level: int = 6, upload_to_cloud: bool = False, disable_progress: bool = False )` Download filing to local storage. **Parameters:** - `data_directory`: Download directory (defaults to Edgar data directory) - `compress`: Compress downloaded files (default: True) - `compression_level`: gzip level 1-9 (default: 6) - `upload_to_cloud`: Upload to cloud storage after download - `disable_progress`: Disable progress display **Example:** `# Download with defaults filing.download() # Custom directory without compression filing.download(data_directory="./raw_filings", compress=False)` Common Recipes -------------- ### Extract revenue from 10-K `from edgar import Company company = Company("AAPL") filings = get_filings(2024, 1, form="10-K") filing = filings.latest() # Get TenK object tenk = filing.obj() # Access financials if tenk.financials: income = tenk.financials.income_statement print(income)` ### Search across multiple filings `from edgar import get_filings filings = get_filings(2024, 1, form="8-K").head(100) for filing in filings: results = filing.search("cybersecurity") if results: print(f"{filing.company} ({filing.filing_date}): {len(results)} mentions")` ### Download exhibits from a filing `filing = get_by_accession_number("0001234567-24-000001") for exhibit in filing.exhibits: print(f"Downloading {exhibit.exhibit_number}: {exhibit.description}") exhibit.download(f"./exhibits/{exhibit.document}")` ### Convert filing to markdown for analysis `filing = company.get_filings(form="10-K").latest() # Export to markdown md = filing.markdown(include_page_breaks=True) # Save for LLM processing with open("filing_for_analysis.md", "w") as f: f.write(md)` Error Handling -------------- `try: filing = get_by_accession_number("0000320193-23-000106") # Check content availability html = filing.html() if html is None: print("HTML not available") # Check XBRL availability xbrl = filing.xbrl() if xbrl is None: print("No XBRL data") # Get structured object obj = filing.obj() except Exception as e: print(f"Error: {e}")` Performance Tips ---------------- 1. **Check before accessing** - Test for None before processing optional data 2. **Use obj() for structured data** - More efficient than parsing HTML 3. **Cache expensive operations** - Store results of xbrl(), text(), etc. 4. **Filter attachments** - Use `exhibits` property instead of filtering all attachments **Efficient pattern:** `# Get structured object once obj = filing.obj() # Check before using if obj and obj.financials: income = obj.financials.income_statement # Process income statement` See Also -------- * **[Filings API Reference](https://edgartools.readthedocs.io/en/latest/api/filings/) ** - Working with filing collections * **[Company API Reference](https://edgartools.readthedocs.io/en/latest/api/company/) ** - Company-specific filing access * **[XBRL API Reference](https://edgartools.readthedocs.io/en/latest/api/xbrl/) ** - XBRL data extraction * **[Working with Filings Guide](https://edgartools.readthedocs.io/en/latest/guides/working-with-filing/) ** - Practical filing operations * **[Extract Financial Statements](https://edgartools.readthedocs.io/en/latest/guides/extract-statements/) ** - Getting financial data * **[Filing Attachments Guide](https://edgartools.readthedocs.io/en/latest/guides/filing-attachments/) ** - Working with documents and exhibits Back to top --- # Local Storage - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/local-storage/#local-storage-guide) Local Storage Guide =================== **edgartools** is designed for interactive queries against **SEC Edgar**, which means it normally makes HTTP requests to the SEC website to retrieve data. For example, when you call `company.submissions` or `filing.attachments`, it makes a request to the SEC. There are times when you want to minimize or eliminate these requests: 1. **Performance**: Speed up processing by avoiding network requests 2. **Offline usage**: Work without internet access or in restricted environments 3. **Bandwidth efficiency**: Reduce data usage and respect SEC rate limits 4. **Development**: Use cached data for testing and development **edgartools** provides comprehensive local storage capabilities to address these needs. Understanding What Gets Downloaded ---------------------------------- Important: Metadata vs Filing Content **edgartools has two separate download functions that serve different purposes:** | Function | What it downloads | Size | Use case | | --- | --- | --- | --- | | `download_edgar_data()` | **Metadata only**: company info, filing indexes, financial facts from SEC's bulk APIs | ~24 GB | Company lookups, `EntityFacts` financials, browsing filing lists | | `download_filings()` | **Actual filing documents**: the complete filing content including XBRL files | Varies | Parsing `filing.xbrl()`, reading filing text, document analysis | **Common misconception**: Running `download_edgar_data()` does NOT give you offline access to XBRL data from individual filings. The financial facts from `download_edgar_data()` come from SEC's pre-processed CompanyFacts API, which is different from parsing XBRL directly from filings. **For offline XBRL access**, you must also run `download_filings()` to download the actual filing documents. ### Quick Reference: What Do I Need? | I want to... | Function needed | | --- | --- | | Look up companies by ticker/CIK offline | Already works for ~10,600 tickers (bundled data). For the full universe: `download_edgar_data(reference=True)` | | Use `company.get_facts()` / `EntityFacts` offline | `download_edgar_data(facts=True)` | | Browse filing lists offline | `download_edgar_data(submissions=True)` | | Parse `filing.xbrl()` offline | `download_filings()` | | Read filing HTML/text offline | `download_filings()` | | Analyze filing attachments offline | `download_filings()` | Bundled Reference Data (Always Available) ----------------------------------------- Ticker lookups work offline by default edgartools ships with a bundled `company_tickers.parquet` file containing ~10,600 exchange-listed tickers with CIK, ticker, company name, and exchange. This is loaded automatically — no download or configuration needed. `Company("AAPL")` works without an internet connection. See [How Ticker Resolution Works](https://edgartools.readthedocs.io/en/latest/guides/finding-companies/#how-ticker-resolution-works) for details. To get the full SEC ticker universe (including recent IPOs and non-exchange entities), download reference data as described below. Supported Local Data Types -------------------------- | Data Type | Description | | --- | --- | | **Company Submissions** | Company metadata and their 1000 most recent filings | | **Company Facts** | Standardized company financial facts from XBRL filings | | **Filing Attachments** | Complete filing documents with all attachments | | **Reference Data** | Company and mutual fund tickers, exchanges, and other lookups | Local Data Directory -------------------- ### Default Location The default local data directory is: `/.edgar` ### Setting the Directory You can set the local data directory in several ways: **Method 1: Environment Variable** `export EDGAR_LOCAL_DATA_DIR="/path/to/local/data"` **Method 2: Programmatic (New)** `from edgar import set_local_storage_path import os # Create the directory first os.makedirs("/tmp/edgar_data", exist_ok=True) # Set the path set_local_storage_path("/tmp/edgar_data")` **Method 3: One-Step Setup (New)** `from edgar import use_local_storage import os # Create and set directory, enable local storage in one call os.makedirs("/tmp/edgar_data", exist_ok=True) use_local_storage("/tmp/edgar_data")` Enabling Local Storage ---------------------- ### Basic Usage `from edgar import use_local_storage import os # Create directory os.makedirs("~/Documents/edgar", exist_ok=True) # Set path and enable in one call use_local_storage("~/Documents/edgar")` ### All Supported Patterns `# 1. BACKWARD COMPATIBLE use_local_storage(True) # Enable use_local_storage(False) # Disable use_local_storage() # Enable (default) # 2. NEW INTUITIVE SYNTAX use_local_storage("/tmp/edgar_data") # Path as string use_local_storage("~/Documents/edgar") # Tilde expansion use_local_storage(Path.home() / "edgar") # Path object # 3. ADVANCED CONTROL use_local_storage("/tmp/edgar", True) # Set path and enable use_local_storage("/tmp/edgar", False) # Set path but keep disabled` ### Checking Status `from edgar import is_using_local_storage if is_using_local_storage(): print("Local storage is enabled") else: print("Using remote SEC data")` Downloading Data to Local Storage --------------------------------- ### Download Bulk SEC Data You can download bulk SEC data using the `download_edgar_data()` function: `from edgar import download_edgar_data # Download all data types (submissions, facts, reference data) download_edgar_data() # Download only specific data types download_edgar_data( submissions=True, # Company metadata and recent filings facts=True, # Company financial facts reference=True # Tickers, exchanges, etc. )` ### Download Complete Filings Download individual filings with all attachments using `download_filings()`: `from edgar import download_filings # Download all filings for a specific date download_filings("2025-01-15") # Download filings for a date range download_filings("2025-01-01:2025-01-15") # Download from a start date onwards download_filings("2025-01-01:") # Download up to an end date download_filings(":2025-01-15")` **Note:** Downloaded filings are stored in `EDGAR_LOCAL_DATA_DIR/filings/YYYYMMDD/`. When local storage is enabled, edgartools automatically checks local storage first before making SEC requests. Complete Workflow Examples -------------------------- ### Example 1: Quick Setup for Development `from edgar import use_local_storage, download_edgar_data import os # Setup local storage in one command os.makedirs("~/edgar_dev", exist_ok=True) use_local_storage("~/edgar_dev") # Download essential data download_edgar_data(submissions=True, reference=True) # Now all queries use local data when available from edgar import Company apple = Company("AAPL") # Uses local data` ### Example 2: High-Performance Analysis Setup `from edgar import use_local_storage, download_filings, get_filings import os # Setup high-performance storage os.makedirs("/tmp/edgar_fast", exist_ok=True) use_local_storage("/tmp/edgar_fast") # Download specific filings for analysis filings = get_filings(form="10-K", year=2024) download_filings(filings=filings) # All subsequent operations are lightning fast for filing in filings: financial_data = filing.xbrl() # Instant from local storage` ### Example 3: Offline Research Environment `from edgar import use_local_storage, download_edgar_data, download_filings import os # Setup offline-capable environment os.makedirs("~/research/edgar_offline", exist_ok=True) use_local_storage("~/research/edgar_offline") # Step 1: Download metadata (company info, filing indexes, facts API data) download_edgar_data() # ~24 GB - enables company lookups and EntityFacts # Step 2: Download actual filing documents for XBRL parsing # This is required if you want to use filing.xbrl() offline! download_filings("2024-01-01:2024-12-31") # Full year of filings # Now works completely offline from edgar import Company, get_filings # These work with just download_edgar_data(): company = Company("AAPL") # Company lookup facts = company.get_facts() # EntityFacts from bulk API filings = get_filings(form="10-K") # Filing list browsing # This requires download_filings(): for filing in filings: xbrl = filing.xbrl() # Parses actual filing document` Storage Planning * `download_edgar_data()`: ~24 GB one-time download * `download_filings()`: ~100-500 MB per day of filings * A full year of filings: ~50-150 GB depending on form types Filtering Downloads ------------------- ### Download Specific Filings Instead of downloading all filings, you can filter to download only what you need: `from edgar import get_filings, download_filings # Get filings with filters filings = get_filings(form="10-K", year=2024).filter(exchange="NYSE") # Download only these filtered filings download_filings(filings=filings)` ### Advanced Filtering Examples `# Download only tech companies' 10-K filings tech_filings = (get_filings(form="10-K", year=2024) .filter(exchange=["NASDAQ", "NYSE"]) download_filings(filings=tech_filings) # Download recent 8-K filings for specific analysis recent_8k = get_filings(form="8-K", filing_date="2025-01-01:") download_filings(filings=recent_8k)` Performance Considerations -------------------------- ### Storage Space Different data types require different amounts of storage: | Data Type | Typical Size | Description | | --- | --- | --- | | Reference Data | ~50 MB | Tickers, exchanges, mappings | | Company Facts | ~2 GB | Compressed financial facts | | Submissions | ~5 GB | Company metadata and filings | | Daily Filings | ~100-500 MB | All filings for one day | ### Download Times * **Reference data**: 1-2 seconds * **Company facts**: 2-3 minutes * **Company submissions**: 2-3 minutes * **Daily filings**: 3-15 minutes depending on volume and time of day ### Compression Filings are automatically compressed to save space. The default compression level is set to 6, but you can adjust it to 9 for maximum compression: `# Enable compression during download download_filings("2025-01-15", compression_level=9)` Directory Structure ------------------- When using local storage, data is organized as follows: `EDGAR_LOCAL_DATA_DIR/ ├── reference/ # Ticker and exchange data │ ├── company_tickers.json │ ├── ticker.txt │ └── ... ├── companyfacts/ # Company financial facts │ └── CIK[0-9]*.json ├── submissions/ # Company submission data │ └── CIK[0-9]*.json └── filings/ # Individual filing documents ├── 20250115/ # Filings by date │ ├── filing1.nc # SGML filing documents │ └── filing2.nc.gz # Compressed filings └── ...` Best Practices -------------- ### 1\. Start Small `# Begin with reference data only use_local_storage("~/edgar_test") download_edgar_data(submissions=False, facts=False, reference=True)` ### 2\. Use Appropriate Storage `# Fast SSD for high-performance analysis use_local_storage("/fast_ssd/edgar_data") # Network storage for shared team access use_local_storage("/shared/team/edgar_data")` ### 3\. Monitor Storage Usage `from edgar.core import get_edgar_data_directory import shutil storage_path = get_edgar_data_directory() total, used, free = shutil.disk_usage(storage_path) print(f"Storage usage: {used // (1024**3)} GB used, {free // (1024**3)} GB free")` ### 4\. Incremental Updates `# Download only recent filings regularly from datetime import datetime, timedelta recent_date = (datetime.now() - timedelta(days=7)).strftime("%Y-%m-%d") download_filings(f"{recent_date}:")` Troubleshooting --------------- ### Common Issues **Path doesn't exist:** `# ❌ This will fail use_local_storage("/nonexistent/path") # ✅ Create directory first import os os.makedirs("/tmp/edgar", exist_ok=True) use_local_storage("/tmp/edgar")` **Insufficient space:** `# Check available space before large downloads import shutil _, _, free = shutil.disk_usage("/tmp") if free < 10 * (1024**3): # 10 GB print("Warning: Less than 10 GB free space")` **Mixed local/remote data:** `# Ensure consistent data source from edgar import is_using_local_storage if is_using_local_storage(): print("Using local storage") else: print("Using remote SEC data") # Enable if needed: use_local_storage(True)` **Network timeout when calling `filing.xbrl()` after `download_edgar_data()`:** This is a common misconception. `download_edgar_data()` only downloads metadata (company info, filing indexes, facts API data). It does **not** download the actual filing documents needed for `filing.xbrl()`. `# ❌ This won't work offline - download_edgar_data() doesn't include filing content download_edgar_data() filing = get_filings()[0] xbrl = filing.xbrl() # Still needs network access! # ✅ You need to also download the filing documents download_edgar_data() # Metadata download_filings("2024-01-01:") # Actual filing documents filing = get_filings()[0] xbrl = filing.xbrl() # Now works offline!` **Alternative: Use EntityFacts for offline financial data** If you only need standard financial metrics and don't need to parse raw XBRL, `EntityFacts` works with just `download_edgar_data()`: `download_edgar_data(facts=True) # Just the facts API data company = Company("AAPL") facts = company.get_facts() # Works offline! income = facts.income_statement() # Pre-processed financials` Migration and Backup -------------------- ### Moving Local Storage `# Old location old_path = "~/old_edgar" # New location new_path = "/new/location/edgar" os.makedirs(new_path, exist_ok=True) # Move data (outside Python) # cp -r ~/old_edgar/* /new/location/edgar/ # Update edgartools use_local_storage(new_path)` ### Backup Strategy `# Create backup of critical data import shutil from datetime import datetime backup_name = f"edgar_backup_{datetime.now().strftime('%Y%m%d')}" shutil.copytree(get_edgar_data_directory(), f"/backups/{backup_name}")` Summary ------- The enhanced local storage system in edgartools provides: ### Key Functions * **`use_local_storage()`**: Enable/disable local storage with optional path setting * **`set_local_storage_path()`**: Set storage directory path * **`is_using_local_storage()`**: Check current status * **`download_edgar_data()`**: Download bulk SEC data * **`download_filings()`**: Download individual filings ### Benefits * **Dramatic performance improvements**: 10-100x faster than remote requests * **Offline capability**: Work without internet connectivity * **Bandwidth efficiency**: Reduce network usage and respect SEC limits * **Flexible configuration**: Multiple ways to configure and use * **Comprehensive data**: Support for all major SEC data types ### Quick Start `import os from edgar import use_local_storage, download_edgar_data # Setup in one line os.makedirs("~/edgar", exist_ok=True) use_local_storage("~/edgar") # Download essential data download_edgar_data() # All subsequent operations use local storage when available` This comprehensive local storage system makes edgartools significantly more efficient for both development and production use cases. Back to top --- # Filing - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/api/filing/#filing-api-reference-access-sec-filing-content-xbrl-data-and-documents) Filing API Reference — Access SEC Filing Content, XBRL Data, and Documents ========================================================================== The `Filing` class represents a single SEC filing and provides comprehensive access to its content, structured data, attachments, and metadata. **Quick example:** `from edgar import get_by_accession_number filing = get_by_accession_number("0000320193-23-000106") print(f"{filing.company} {filing.form} filed {filing.filing_date}") # Access content text = filing.text() xbrl = filing.xbrl() # Get form-specific object tenk = filing.obj() # Returns TenK for 10-K filings` Getting a Filing ---------------- `from edgar import Company, get_filings, get_by_accession_number # From a company company = Company("AAPL") filing = company.get_filings(form="10-K").latest() # From a search filings = get_filings(2024, 1, form="10-K") filing = filings[0] # By accession number filing = get_by_accession_number("0000320193-23-000106")` Core Properties --------------- ### Basic Information | Property | Type | Description | | --- | --- | --- | | `cik` | int | Central Index Key of the filing entity | | `company` | str | Company name | | `form` | str | SEC form type (e.g., "10-K", "10-Q", "8-K") | | `filing_date` | str | Date filed with SEC (YYYY-MM-DD) | | `accession_no` | str | Unique SEC accession number | | `accession_number` | str | Alias for `accession_no` | | `period_of_report` | str | Reporting period end date | **Example:** `print(filing.cik) # 320193 print(filing.company) # "Apple Inc." print(filing.form) # "10-K" print(filing.filing_date) # "2023-11-03" print(filing.period_of_report) # "2023-09-30"` ### Document Properties | Property | Type | Description | | --- | --- | --- | | `document` | Attachment | Primary display document | | `primary_documents` | List\[Attachment\] | All primary documents | | `attachments` | Attachments | All documents and attachments | | `exhibits` | Attachments | Exhibits only (subset of attachments) | **Example:** `# Access primary document doc = filing.document print(doc.document_type) # Loop through all attachments for att in filing.attachments: print(f"{att.sequence}: {att.description}") # Access exhibits for exhibit in filing.exhibits: print(f"Exhibit {exhibit.exhibit_number}: {exhibit.description}")` ### URL Properties | Property | Description | | --- | --- | | `homepage_url` | Filing homepage on SEC website | | `filing_url` | URL to primary filing document | | `text_url` | URL to text version | | `base_dir` | Base directory for all filing files | **Example:** `print(filing.homepage_url) # https://www.sec.gov/Archives/edgar/data/320193/000032019323000106/0000320193-23-000106-index.html print(filing.filing_url) # https://www.sec.gov/Archives/edgar/data/320193/000032019323000106/aapl-20230930.htm` ### Metadata Properties | Property | Type | Description | | --- | --- | --- | | `header` | FilingHeader | Parsed SGML header information | | `is_multi_entity` | bool | Whether filing involves multiple entities | | `all_ciks` | List\[int\] | All CIK numbers in filing | | `all_entities` | List\[str\] | All entity names in filing | Content Access Methods ---------------------- ### Raw Content #### html() `def html(self) -> Optional[str]` Get HTML content of the primary document. **Returns:** HTML string or None if not available **Example:** `html = filing.html() if html: print(f"HTML length: {len(html)} characters")` #### text() `def text(self) -> str` Convert filing HTML to clean plain text. **Returns:** Plain text content **Example:** `text = filing.text() # Search within text if "artificial intelligence" in text.lower(): print("AI mentioned in filing")` #### markdown() `def markdown( include_page_breaks: bool = False, start_page_number: int = 0 ) -> str` Convert filing to Markdown format. **Parameters:** - `include_page_breaks` (bool): Include page break markers - `start_page_number` (int): Starting page number for page breaks **Returns:** Markdown formatted content **Example:** `md = filing.markdown() with open("filing.md", "w") as f: f.write(md)` #### xml() `def xml(self) -> Optional[str]` Get XML content if filing contains XML data. **Returns:** XML string or None **Example:** `xml = filing.xml() if xml: import xml.etree.ElementTree as ET root = ET.fromstring(xml)` #### full\_text\_submission() `def full_text_submission(self) -> str` Get the complete SEC text submission file. **Returns:** Full submission text including SGML headers ### Structured Data Access #### xbrl() `def xbrl(self) -> Optional[XBRL]` Get XBRL data object if filing contains XBRL. **Returns:** `XBRL` object or None **Example:** `xbrl = filing.xbrl() if xbrl: # Access financial statements income = xbrl.statements.income_statement() balance = xbrl.statements.balance_sheet() cashflow = xbrl.statements.cash_flow_statement()` **See also:** [XBRL API Reference](https://edgartools.readthedocs.io/en/stable/api/xbrl/) , [Extract Financial Statements Guide](https://edgartools.readthedocs.io/en/stable/guides/extract-statements/) #### obj() / data\_object() `def obj(self) def data_object(self) # Alias` Get form-specific structured object based on filing type. **Returns:** Form-specific object (TenK, TenQ, EightK, Form4, etc.) **Form type mappings:** | Form Type | Return Class | Module | | --- | --- | --- | | 10-K | TenK | edgar.company\_reports | | 10-Q | TenQ | edgar.company\_reports | | 8-K | EightK | edgar.company\_reports | | 20-F | TwentyF | edgar.company\_reports | | 4 | Form4 | edgar.ownership | | 3 | Form3 | edgar.ownership | | 5 | Form5 | edgar.ownership | | DEF 14A | ProxyStatement | edgar.proxy | | 13F-HR | ThirteenF | edgar.holdings | | SC 13D/G | Schedule13 | edgar.ownership | | NPORT-P | NportFiling | edgar.nport | | 144 | Form144 | edgar.ownership | **Example:** `# For a 10-K filing tenk = filing.obj() print(type(tenk)) # # Access financial statements from TenK object if tenk.financials: income = tenk.financials.income_statement balance = tenk.financials.balance_sheet cashflow = tenk.financials.cash_flow_statement # Or use direct properties income = tenk.income_statement balance = tenk.balance_sheet # XBRL report pages (also available via filing.reports) reports = tenk.reports` **Important:** The base `Filing` class does **not** have a `financials` property. To access financial data: - Use `filing.obj().financials` for 10-K/10-Q filings - Or use `filing.xbrl().statements` for any XBRL filing **Incorrect:** `# This will fail - Filing has no financials property financials = filing.financials # AttributeError` **Correct:** `# Get form-specific object first tenk = filing.obj() if tenk.financials: financials = tenk.financials # Or use XBRL directly xbrl = filing.xbrl() if xbrl: statements = xbrl.statements` ### Parsing and Search #### parse() `def parse(self) -> Document` Parse filing into structured Document for advanced searching. **Returns:** Parsed `Document` object **Example:** `doc = filing.parse() # Use document methods for structured search` #### search() `def search(self, query: str, regex: bool = False) -> List[str]` Search for text within filing content. **Parameters:** - `query` (str): Search term or pattern - `regex` (bool): Treat query as regex pattern **Returns:** List of matching text excerpts **Example:** `# Simple text search results = filing.search("revenue recognition") print(f"Found {len(results)} mentions") # Regex search for emails emails = filing.search(r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b', regex=True)` #### sections() `def sections(self) -> List[str]` Get list of available document sections. **Returns:** List of section names/identifiers **Example:** `sections = filing.sections() for section in sections: print(section) # "Item 1", "Item 2", etc.` #### sgml() `def sgml(self) -> FilingSGML` Get parsed SGML structure of filing. **Returns:** `FilingSGML` object with document structure **Example:** `sgml = filing.sgml() for doc in sgml.documents: print(f"{doc.type}: {doc.sequence}")` Interactive Methods ------------------- ### Viewing and Display #### view() `def view(self)` Display filing in console or Jupyter notebook with Rich formatting. **Example:** `filing.view() # Shows formatted filing content` #### open() `def open(self)` Open primary filing document in default web browser. **Example:** `filing.open() # Opens filing in browser` #### open\_homepage() `def open_homepage(self)` Open filing homepage (index page) in default web browser. **Example:** `filing.open_homepage() # Opens SEC filing index page` #### serve() `def serve(self, port: int = 8000)` Serve filing on local HTTP server for viewing. **Parameters:** - `port` (int): Server port (default: 8000) **Example:** `filing.serve(port=8080) # Access at http://localhost:8080` Entity and Related Data ----------------------- ### get\_entity() `def get_entity(self) -> Union[Company, Entity]` Get Company or Entity object for this filing. **Returns:** `Company` or `Entity` instance **Example:** `entity = filing.get_entity() print(f"Entity: {entity.name}") print(f"Industry: {entity.industry}")` ### as\_company\_filing() `def as_company_filing(self) -> EntityFiling` Convert to EntityFiling with enhanced metadata. **Returns:** `EntityFiling` object ### related\_filings() `def related_filings(self) -> Filings` Get filings related by file number. **Returns:** `Filings` collection Persistence and Serialization ----------------------------- ### Save and Load #### save() `def save(self, directory_or_file: PathLike)` Save filing using pickle serialization. **Parameters:** - `directory_or_file`: Directory or file path **Example:** `# Save to directory filing.save("./data/filings/") # Save to specific file filing.save("./data/apple_10k_2023.pkl")` #### load() `@classmethod def load(cls, path: PathLike) -> Filing` Load filing from pickle file. **Parameters:** - `path`: Path to pickle file **Returns:** `Filing` object **Example:** `filing = Filing.load("./data/apple_10k_2023.pkl")` ### Data Export #### to\_dict() `def to_dict(self) -> Dict[str, Union[str, int]]` Convert to dictionary representation. **Returns:** Dictionary with filing data **Example:** `data = filing.to_dict() print(data.keys()) # dict_keys(['cik', 'company', 'form', 'filing_date', 'accession_no', ...])` #### from\_dict() `@classmethod def from_dict(cls, data: Dict) -> Filing` Create Filing from dictionary. **Parameters:** - `data`: Dictionary with filing information **Returns:** `Filing` object #### summary() `def summary(self) -> pd.DataFrame` Get filing summary as pandas DataFrame. **Returns:** DataFrame with filing metadata #### to\_context() `def to_context(self, detail: str) -> str` Generate context string for LLM/AI use. **Parameters:** - `detail` (str): Level of detail **Returns:** Context string ### Download #### download() `def download( self, data_directory: Optional[str] = None, compress: bool = True, compression_level: int = 6, upload_to_cloud: bool = False, disable_progress: bool = False )` Download filing to local storage. **Parameters:** - `data_directory`: Download directory (defaults to Edgar data directory) - `compress`: Compress downloaded files (default: True) - `compression_level`: gzip level 1-9 (default: 6) - `upload_to_cloud`: Upload to cloud storage after download - `disable_progress`: Disable progress display **Example:** `# Download with defaults filing.download() # Custom directory without compression filing.download(data_directory="./raw_filings", compress=False)` Common Recipes -------------- ### Extract revenue from 10-K `from edgar import Company company = Company("AAPL") filings = get_filings(2024, 1, form="10-K") filing = filings.latest() # Get TenK object tenk = filing.obj() # Access financials if tenk.financials: income = tenk.financials.income_statement print(income)` ### Search across multiple filings `from edgar import get_filings filings = get_filings(2024, 1, form="8-K").head(100) for filing in filings: results = filing.search("cybersecurity") if results: print(f"{filing.company} ({filing.filing_date}): {len(results)} mentions")` ### Download exhibits from a filing `filing = get_by_accession_number("0001234567-24-000001") for exhibit in filing.exhibits: print(f"Downloading {exhibit.exhibit_number}: {exhibit.description}") exhibit.download(f"./exhibits/{exhibit.document}")` ### Convert filing to markdown for analysis `filing = company.get_filings(form="10-K").latest() # Export to markdown md = filing.markdown(include_page_breaks=True) # Save for LLM processing with open("filing_for_analysis.md", "w") as f: f.write(md)` Error Handling -------------- `try: filing = get_by_accession_number("0000320193-23-000106") # Check content availability html = filing.html() if html is None: print("HTML not available") # Check XBRL availability xbrl = filing.xbrl() if xbrl is None: print("No XBRL data") # Get structured object obj = filing.obj() except Exception as e: print(f"Error: {e}")` Performance Tips ---------------- 1. **Check before accessing** - Test for None before processing optional data 2. **Use obj() for structured data** - More efficient than parsing HTML 3. **Cache expensive operations** - Store results of xbrl(), text(), etc. 4. **Filter attachments** - Use `exhibits` property instead of filtering all attachments **Efficient pattern:** `# Get structured object once obj = filing.obj() # Check before using if obj and obj.financials: income = obj.financials.income_statement # Process income statement` See Also -------- * **[Filings API Reference](https://edgartools.readthedocs.io/en/stable/api/filings/) ** - Working with filing collections * **[Company API Reference](https://edgartools.readthedocs.io/en/stable/api/company/) ** - Company-specific filing access * **[XBRL API Reference](https://edgartools.readthedocs.io/en/stable/api/xbrl/) ** - XBRL data extraction * **[Working with Filings Guide](https://edgartools.readthedocs.io/en/stable/guides/working-with-filing/) ** - Practical filing operations * **[Extract Financial Statements](https://edgartools.readthedocs.io/en/stable/guides/extract-statements/) ** - Getting financial data * **[Filing Attachments Guide](https://edgartools.readthedocs.io/en/stable/guides/filing-attachments/) ** - Working with documents and exhibits Back to top --- # Local Storage - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/local-storage/#local-storage-guide) Local Storage Guide =================== **edgartools** is designed for interactive queries against **SEC Edgar**, which means it normally makes HTTP requests to the SEC website to retrieve data. For example, when you call `company.submissions` or `filing.attachments`, it makes a request to the SEC. There are times when you want to minimize or eliminate these requests: 1. **Performance**: Speed up processing by avoiding network requests 2. **Offline usage**: Work without internet access or in restricted environments 3. **Bandwidth efficiency**: Reduce data usage and respect SEC rate limits 4. **Development**: Use cached data for testing and development **edgartools** provides comprehensive local storage capabilities to address these needs. Understanding What Gets Downloaded ---------------------------------- Important: Metadata vs Filing Content **edgartools has two separate download functions that serve different purposes:** | Function | What it downloads | Size | Use case | | --- | --- | --- | --- | | `download_edgar_data()` | **Metadata only**: company info, filing indexes, financial facts from SEC's bulk APIs | ~24 GB | Company lookups, `EntityFacts` financials, browsing filing lists | | `download_filings()` | **Actual filing documents**: the complete filing content including XBRL files | Varies | Parsing `filing.xbrl()`, reading filing text, document analysis | **Common misconception**: Running `download_edgar_data()` does NOT give you offline access to XBRL data from individual filings. The financial facts from `download_edgar_data()` come from SEC's pre-processed CompanyFacts API, which is different from parsing XBRL directly from filings. **For offline XBRL access**, you must also run `download_filings()` to download the actual filing documents. ### Quick Reference: What Do I Need? | I want to... | Function needed | | --- | --- | | Look up companies by ticker/CIK offline | Already works for ~10,600 tickers (bundled data). For the full universe: `download_edgar_data(reference=True)` | | Use `company.get_facts()` / `EntityFacts` offline | `download_edgar_data(facts=True)` | | Browse filing lists offline | `download_edgar_data(submissions=True)` | | Parse `filing.xbrl()` offline | `download_filings()` | | Read filing HTML/text offline | `download_filings()` | | Analyze filing attachments offline | `download_filings()` | Bundled Reference Data (Always Available) ----------------------------------------- Ticker lookups work offline by default edgartools ships with a bundled `company_tickers.parquet` file containing ~10,600 exchange-listed tickers with CIK, ticker, company name, and exchange. This is loaded automatically — no download or configuration needed. `Company("AAPL")` works without an internet connection. See [How Ticker Resolution Works](https://edgartools.readthedocs.io/en/stable/guides/finding-companies/#how-ticker-resolution-works) for details. To get the full SEC ticker universe (including recent IPOs and non-exchange entities), download reference data as described below. Supported Local Data Types -------------------------- | Data Type | Description | | --- | --- | | **Company Submissions** | Company metadata and their 1000 most recent filings | | **Company Facts** | Standardized company financial facts from XBRL filings | | **Filing Attachments** | Complete filing documents with all attachments | | **Reference Data** | Company and mutual fund tickers, exchanges, and other lookups | Local Data Directory -------------------- ### Default Location The default local data directory is: `/.edgar` ### Setting the Directory You can set the local data directory in several ways: **Method 1: Environment Variable** `export EDGAR_LOCAL_DATA_DIR="/path/to/local/data"` **Method 2: Programmatic (New)** `from edgar import set_local_storage_path import os # Create the directory first os.makedirs("/tmp/edgar_data", exist_ok=True) # Set the path set_local_storage_path("/tmp/edgar_data")` **Method 3: One-Step Setup (New)** `from edgar import use_local_storage import os # Create and set directory, enable local storage in one call os.makedirs("/tmp/edgar_data", exist_ok=True) use_local_storage("/tmp/edgar_data")` Enabling Local Storage ---------------------- ### Basic Usage `from edgar import use_local_storage import os # Create directory os.makedirs("~/Documents/edgar", exist_ok=True) # Set path and enable in one call use_local_storage("~/Documents/edgar")` ### All Supported Patterns `# 1. BACKWARD COMPATIBLE use_local_storage(True) # Enable use_local_storage(False) # Disable use_local_storage() # Enable (default) # 2. NEW INTUITIVE SYNTAX use_local_storage("/tmp/edgar_data") # Path as string use_local_storage("~/Documents/edgar") # Tilde expansion use_local_storage(Path.home() / "edgar") # Path object # 3. ADVANCED CONTROL use_local_storage("/tmp/edgar", True) # Set path and enable use_local_storage("/tmp/edgar", False) # Set path but keep disabled` ### Checking Status `from edgar import is_using_local_storage if is_using_local_storage(): print("Local storage is enabled") else: print("Using remote SEC data")` Downloading Data to Local Storage --------------------------------- ### Download Bulk SEC Data You can download bulk SEC data using the `download_edgar_data()` function: `from edgar import download_edgar_data # Download all data types (submissions, facts, reference data) download_edgar_data() # Download only specific data types download_edgar_data( submissions=True, # Company metadata and recent filings facts=True, # Company financial facts reference=True # Tickers, exchanges, etc. )` ### Download Complete Filings Download individual filings with all attachments using `download_filings()`: `from edgar import download_filings # Download all filings for a specific date download_filings("2025-01-15") # Download filings for a date range download_filings("2025-01-01:2025-01-15") # Download from a start date onwards download_filings("2025-01-01:") # Download up to an end date download_filings(":2025-01-15")` **Note:** Downloaded filings are stored in `EDGAR_LOCAL_DATA_DIR/filings/YYYYMMDD/`. When local storage is enabled, edgartools automatically checks local storage first before making SEC requests. Complete Workflow Examples -------------------------- ### Example 1: Quick Setup for Development `from edgar import use_local_storage, download_edgar_data import os # Setup local storage in one command os.makedirs("~/edgar_dev", exist_ok=True) use_local_storage("~/edgar_dev") # Download essential data download_edgar_data(submissions=True, reference=True) # Now all queries use local data when available from edgar import Company apple = Company("AAPL") # Uses local data` ### Example 2: High-Performance Analysis Setup `from edgar import use_local_storage, download_filings, get_filings import os # Setup high-performance storage os.makedirs("/tmp/edgar_fast", exist_ok=True) use_local_storage("/tmp/edgar_fast") # Download specific filings for analysis filings = get_filings(form="10-K", year=2024) download_filings(filings=filings) # All subsequent operations are lightning fast for filing in filings: financial_data = filing.xbrl() # Instant from local storage` ### Example 3: Offline Research Environment `from edgar import use_local_storage, download_edgar_data, download_filings import os # Setup offline-capable environment os.makedirs("~/research/edgar_offline", exist_ok=True) use_local_storage("~/research/edgar_offline") # Step 1: Download metadata (company info, filing indexes, facts API data) download_edgar_data() # ~24 GB - enables company lookups and EntityFacts # Step 2: Download actual filing documents for XBRL parsing # This is required if you want to use filing.xbrl() offline! download_filings("2024-01-01:2024-12-31") # Full year of filings # Now works completely offline from edgar import Company, get_filings # These work with just download_edgar_data(): company = Company("AAPL") # Company lookup facts = company.get_facts() # EntityFacts from bulk API filings = get_filings(form="10-K") # Filing list browsing # This requires download_filings(): for filing in filings: xbrl = filing.xbrl() # Parses actual filing document` Storage Planning * `download_edgar_data()`: ~24 GB one-time download * `download_filings()`: ~100-500 MB per day of filings * A full year of filings: ~50-150 GB depending on form types Filtering Downloads ------------------- ### Download Specific Filings Instead of downloading all filings, you can filter to download only what you need: `from edgar import get_filings, download_filings # Get filings with filters filings = get_filings(form="10-K", year=2024).filter(exchange="NYSE") # Download only these filtered filings download_filings(filings=filings)` ### Advanced Filtering Examples `# Download only tech companies' 10-K filings tech_filings = (get_filings(form="10-K", year=2024) .filter(exchange=["NASDAQ", "NYSE"]) download_filings(filings=tech_filings) # Download recent 8-K filings for specific analysis recent_8k = get_filings(form="8-K", filing_date="2025-01-01:") download_filings(filings=recent_8k)` Performance Considerations -------------------------- ### Storage Space Different data types require different amounts of storage: | Data Type | Typical Size | Description | | --- | --- | --- | | Reference Data | ~50 MB | Tickers, exchanges, mappings | | Company Facts | ~2 GB | Compressed financial facts | | Submissions | ~5 GB | Company metadata and filings | | Daily Filings | ~100-500 MB | All filings for one day | ### Download Times * **Reference data**: 1-2 seconds * **Company facts**: 2-3 minutes * **Company submissions**: 2-3 minutes * **Daily filings**: 3-15 minutes depending on volume and time of day ### Compression Filings are automatically compressed to save space. The default compression level is set to 6, but you can adjust it to 9 for maximum compression: `# Enable compression during download download_filings("2025-01-15", compression_level=9)` Directory Structure ------------------- When using local storage, data is organized as follows: `EDGAR_LOCAL_DATA_DIR/ ├── reference/ # Ticker and exchange data │ ├── company_tickers.json │ ├── ticker.txt │ └── ... ├── companyfacts/ # Company financial facts │ └── CIK[0-9]*.json ├── submissions/ # Company submission data │ └── CIK[0-9]*.json └── filings/ # Individual filing documents ├── 20250115/ # Filings by date │ ├── filing1.nc # SGML filing documents │ └── filing2.nc.gz # Compressed filings └── ...` Best Practices -------------- ### 1\. Start Small `# Begin with reference data only use_local_storage("~/edgar_test") download_edgar_data(submissions=False, facts=False, reference=True)` ### 2\. Use Appropriate Storage `# Fast SSD for high-performance analysis use_local_storage("/fast_ssd/edgar_data") # Network storage for shared team access use_local_storage("/shared/team/edgar_data")` ### 3\. Monitor Storage Usage `from edgar.core import get_edgar_data_directory import shutil storage_path = get_edgar_data_directory() total, used, free = shutil.disk_usage(storage_path) print(f"Storage usage: {used // (1024**3)} GB used, {free // (1024**3)} GB free")` ### 4\. Incremental Updates `# Download only recent filings regularly from datetime import datetime, timedelta recent_date = (datetime.now() - timedelta(days=7)).strftime("%Y-%m-%d") download_filings(f"{recent_date}:")` Troubleshooting --------------- ### Common Issues **Path doesn't exist:** `# ❌ This will fail use_local_storage("/nonexistent/path") # ✅ Create directory first import os os.makedirs("/tmp/edgar", exist_ok=True) use_local_storage("/tmp/edgar")` **Insufficient space:** `# Check available space before large downloads import shutil _, _, free = shutil.disk_usage("/tmp") if free < 10 * (1024**3): # 10 GB print("Warning: Less than 10 GB free space")` **Mixed local/remote data:** `# Ensure consistent data source from edgar import is_using_local_storage if is_using_local_storage(): print("Using local storage") else: print("Using remote SEC data") # Enable if needed: use_local_storage(True)` **Network timeout when calling `filing.xbrl()` after `download_edgar_data()`:** This is a common misconception. `download_edgar_data()` only downloads metadata (company info, filing indexes, facts API data). It does **not** download the actual filing documents needed for `filing.xbrl()`. `# ❌ This won't work offline - download_edgar_data() doesn't include filing content download_edgar_data() filing = get_filings()[0] xbrl = filing.xbrl() # Still needs network access! # ✅ You need to also download the filing documents download_edgar_data() # Metadata download_filings("2024-01-01:") # Actual filing documents filing = get_filings()[0] xbrl = filing.xbrl() # Now works offline!` **Alternative: Use EntityFacts for offline financial data** If you only need standard financial metrics and don't need to parse raw XBRL, `EntityFacts` works with just `download_edgar_data()`: `download_edgar_data(facts=True) # Just the facts API data company = Company("AAPL") facts = company.get_facts() # Works offline! income = facts.income_statement() # Pre-processed financials` Migration and Backup -------------------- ### Moving Local Storage `# Old location old_path = "~/old_edgar" # New location new_path = "/new/location/edgar" os.makedirs(new_path, exist_ok=True) # Move data (outside Python) # cp -r ~/old_edgar/* /new/location/edgar/ # Update edgartools use_local_storage(new_path)` ### Backup Strategy `# Create backup of critical data import shutil from datetime import datetime backup_name = f"edgar_backup_{datetime.now().strftime('%Y%m%d')}" shutil.copytree(get_edgar_data_directory(), f"/backups/{backup_name}")` Summary ------- The enhanced local storage system in edgartools provides: ### Key Functions * **`use_local_storage()`**: Enable/disable local storage with optional path setting * **`set_local_storage_path()`**: Set storage directory path * **`is_using_local_storage()`**: Check current status * **`download_edgar_data()`**: Download bulk SEC data * **`download_filings()`**: Download individual filings ### Benefits * **Dramatic performance improvements**: 10-100x faster than remote requests * **Offline capability**: Work without internet connectivity * **Bandwidth efficiency**: Reduce network usage and respect SEC limits * **Flexible configuration**: Multiple ways to configure and use * **Comprehensive data**: Support for all major SEC data types ### Quick Start `import os from edgar import use_local_storage, download_edgar_data # Setup in one line os.makedirs("~/edgar", exist_ok=True) use_local_storage("~/edgar") # Download essential data download_edgar_data() # All subsequent operations use local storage when available` This comprehensive local storage system makes edgartools significantly more efficient for both development and production use cases. Back to top --- # Performance Optimization - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/resources/performance/#performance-optimization) Performance Optimization ======================== Working with SEC data can be resource-intensive due to the volume of data, network latency, and SEC's rate limits. This guide provides strategies to optimize your edgartools workflows for better performance. Understanding How edgartools Fetches Data ----------------------------------------- To optimize performance, it's important to understand how edgartools retrieves data from the SEC EDGAR system. ### How `get_filings()` Works The global `get_filings()` function operates as follows: * It fetches quarterly filing indexes to cover the requested time period * For the current year, it fetches complete data for the year to date * For multiple years, it fetches quarterly indexes for each year * Each quarterly index requires a separate HTTP request For example, requesting filings for 2024 requires 4 HTTP requests (one for each quarter), while requesting filings for 2020-2024 requires 20 HTTP requests. `# This makes 4 HTTP requests (one per quarter) filings_2024 = get_filings(year=2024) # This makes 20 HTTP requests (5 years × 4 quarters) filings_multi_year = get_filings(start_date="2020-01-01", end_date="2024-12-31")` ### How `company.get_filings()` Works The `company.get_filings()` method works differently: * It fetches the company's submission JSON file, which contains all available filings for that company * This requires just one HTTP request, regardless of the date range * The data is then filtered client-side based on your criteria `# This makes just 1 HTTP request, regardless of date range company = Company("AAPL") company_filings = company.get_filings(form="10-K")` ### Filing Content Retrieval Both methods above only return filing metadata (indexes). When you access the actual content of a filing, an additional HTTP request is made: `# This makes an additional HTTP request when you access the filing filing = filings.latest() filing_text = filing.text # HTTP request happens here` Choosing the Right Access Pattern --------------------------------- Based on your specific use case, choose the most efficient access pattern: | If your query is... | Use this approach | Why | | --- | --- | --- | | Focused on specific form types across companies | `get_filings(form="4")` | Efficiently filters by form type | | Focused on a single company | `company.get_filings()` | Makes just one HTTP request | | Across multiple specific companies | `get_filings().filter(cik=["0000320193", "0000789019"])` | Allows precise filtering | | Limited to a specific year | `get_filings(year=2024)` | Minimizes the number of index requests | | Focused on recent filings | `get_filings().latest(100)` | Gets only the most recent filings | Rate Limiting Considerations ---------------------------- By default, edgartools limits requests to a maximum of 10 per second to comply with SEC EDGAR's rate limits. Exceeding these limits can result in your IP being temporarily blocked. `# Default rate limit is 10 requests per second # You can adjust it if needed (use with caution) from edgar import set_rate_limit # Decrease rate limit for more conservative approach set_rate_limit(5) # 5 requests per second` Using Local Storage for Performance ----------------------------------- One of the most effective ways to improve performance is to use local storage. This allows you to: 1. Cache filings locally to avoid repeated HTTP requests 2. Process filings offline without network latency 3. Batch download filings for later analysis ### Setting Up Local Storage `from edgar import enable_local_storage # Enable local storage enable_local_storage("/path/to/storage") # Now filings will be stored locally company = Company("MSFT") filings = company.get_filings(form="10-K") filing = filings.latest() # This will use the local copy if available, or download and cache it if not text = filing.text` ### Batch Downloading Filings For large-scale analysis, batch download filings first, then process them offline: `from edgar import download_filings # Get filing metadata companies = ["AAPL", "MSFT", "GOOGL", "AMZN", "META"] all_filings = [] for ticker in companies: company = Company(ticker) filings = company.get_filings(form="10-K").head(5) # Last 5 10-Ks all_filings.extend(filings) # Batch download all filings (this makes HTTP requests efficiently) download_filings(all_filings, "/path/to/storage") # Now process them offline (no HTTP requests) for filing in all_filings: # Process filing without network latency text = filing.text # Uses local copy` Memory Optimization ------------------- When working with many filings or large filings, memory usage can become a concern. ### Processing Large Datasets For large datasets, use generators and process filings one at a time: `def process_filings_generator(filings): for filing in filings: # Process one filing at a time result = process_filing(filing) yield result # Free memory del filing # Process filings one at a time for result in process_filings_generator(all_filings): save_or_analyze(result)` ### Working with Large Filings For large filings (like 10-Ks), process sections individually: `filing = company.get_latest_filing("10-K").obj() # Process one section at a time sections = ["business", "risk_factors", "management_discussion"] for section_name in sections: if hasattr(filing, section_name): section = getattr(filing, section_name) # Process section process_section(section_name, section) # Free memory del section` Parallel Processing ------------------- For computationally intensive tasks, consider parallel processing: `from concurrent.futures import ThreadPoolExecutor import time def process_filing_with_delay(filing): # Add delay to respect rate limits time.sleep(0.1) # Process filing return {"accession": filing.accession_number, "text_length": len(filing.text)} # Process filings in parallel with a thread pool with ThreadPoolExecutor(max_workers=5) as executor: results = list(executor.map(process_filing_with_delay, all_filings))` Caching Strategies ------------------ Implement caching for expensive operations: `import functools @functools.lru_cache(maxsize=128) def get_filing_sentiment(filing_accession): # Expensive operation to calculate sentiment filing = get_filing_by_accession(filing_accession) text = filing.text # Calculate sentiment (expensive operation) return calculate_sentiment(text) # This will be cached after the first call sentiment = get_filing_sentiment("0000320193-20-000096")` Performance Benchmarks ---------------------- Here are some typical performance benchmarks to help you plan your workflows: | Operation | Typical Time | Notes | | --- | --- | --- | | `get_filings(year=2024)` | 2-5 seconds | Fetches 4 quarterly indexes | | `company.get_filings()` | 1-2 seconds | Single HTTP request | | Downloading a 10-K filing | 1-3 seconds | Depends on filing size | | Parsing a 10-K as Data Object | 2-5 seconds | First-time parsing | | Accessing a locally stored filing | < 0.1 seconds | From disk cache | | Processing 100 filings sequentially | 3-10 minutes | With rate limiting | | Processing 100 filings in parallel | 1-3 minutes | With proper rate limiting | Best Practices Summary ---------------------- 1. **Choose the right access pattern** based on your specific use case 2. **Use `company.get_filings()`** when focusing on a single company 3. **Enable local storage** to avoid repeated HTTP requests 4. **Batch download filings** before processing them 5. **Process filings one at a time** for large datasets 6. **Respect SEC rate limits** to avoid being blocked 7. **Implement caching** for expensive operations 8. **Use parallel processing** carefully with appropriate delays 9. **Filter filings early** in your pipeline to reduce the number of filings to process 10. **Monitor memory usage** when working with large filings or datasets By following these guidelines, you can significantly improve the performance of your edgartools workflows while respecting SEC EDGAR's rate limits and your system's resources. Advanced Techniques ------------------- ### Custom Indexing For repeated analysis of the same dataset, consider creating your own indexes: `import pandas as pd # Create a custom index of filings filings = get_filings(form=["10-K", "10-Q"], year=2024) index_data = [] for filing in filings: index_data.append({ "accession": filing.accession_number, "cik": filing.cik, "company": filing.company_name, "form": filing.form_type, "date": filing.filing_date, "path": filing.get_local_path() if filing.is_local() else None }) # Save as CSV for quick loading index_df = pd.DataFrame(index_data) index_df.to_csv("filings_index_2024.csv", index=False) # Later, load the index instead of fetching again loaded_index = pd.read_csv("filings_index_2024.csv")` ### Incremental Updates For ongoing analysis, implement incremental updates: `import datetime # Get the date of your last update last_update = datetime.date(2024, 6, 1) today = datetime.date.today() # Only fetch filings since your last update new_filings = get_filings(start_date=last_update, end_date=today) # Process only the new filings for filing in new_filings: process_filing(filing) # Update your last update date last_update = today` By implementing these performance optimization strategies, you can make your edgartools workflows more efficient, faster, and more resilient. Back to top --- # Filings - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/api/filings/#filings-api-reference-filter-and-navigate-sec-filing-collections) Filings API Reference — Filter and Navigate SEC Filing Collections ================================================================== The `Filings` class represents a collection of SEC filings with powerful filtering, navigation, and batch processing capabilities. **Quick example:** `from edgar import get_filings # Get Q1 2024 10-K filings filings = get_filings(2024, 1, form="10-K") print(f"Found {len(filings)} filings") # Filter to NASDAQ companies nasdaq = filings.filter(exchange="NASDAQ") # Get latest 10 latest = nasdaq.latest(10) # Export to DataFrame df = latest.to_pandas()` Getting Filings --------------- ### get\_filings() `def get_filings( year: int | List[int] = None, quarter: int | List[int] = None, form: str | List[str] = None, amendments: bool = True, filing_date: str = None ) -> Filings` **Parameters:** - `year` (int | List\[int\]): Calendar year(s) — NOT fiscal year - `quarter` (int | List\[int\]): Calendar quarter(s) 1-4 - `form` (str | List\[str\]): Form type(s) - `amendments` (bool): Include amendments (default: True) - `filing_date` (str): Date filter (YYYY-MM-DD or YYYY-MM-DD:YYYY-MM-DD) **Returns:** `Filings` collection **Important:** There is **no `limit` parameter**. Use `.head(n)` or `.latest(n)` on the result to limit. **Examples:** `from edgar import get_filings # Q1 2024 filings filings = get_filings(2024, 1) # All 2023 10-K filings filings = get_filings(2023, form="10-K") # Multiple years filings = get_filings([2022, 2023, 2024]) # Multiple quarters filings = get_filings(2024, [1, 2], form="10-Q") # Date range filings = get_filings(2024, 1, filing_date="2024-01-01:2024-01-31") # Multiple forms filings = get_filings(2024, 1, form=["10-K", "10-Q"]) # Exclude amendments filings = get_filings(2024, 1, form="10-K", amendments=False)` Core Properties --------------- ### Collection Information | Property | Type | Description | | --- | --- | --- | | `empty` | bool | Whether collection contains any filings | | `date_range` | Tuple\[str, str\] | (start\_date, end\_date) | | `start_date` | str | Earliest filing date | | `end_date` | str | Latest filing date | | `summary` | str | Summary description of collection | **Example:** `if filings.empty: print("No filings found") else: start, end = filings.date_range print(f"Filings from {start} to {end}") print(f"Total: {len(filings)}")` Collection Operations --------------------- ### Size and Access #### len() `len(filings) -> int` Number of filings in collection. **Example:** `print(f"Collection contains {len(filings)} filings")` #### Indexing `filings[index] -> Filing` Access individual filings by index. **Example:** `first = filings[0] last = filings[-1] # Slicing works but returns list, not Filings first_five = filings[:5] # List of Filing objects` #### Iteration `for filing in filings: ...` Iterate through collection. **Example:** `for filing in filings: print(f"{filing.form}: {filing.company} ({filing.filing_date})")` #### get() `def get(self, index_or_accession_number: Union[int, str]) -> Filing` Get filing by index or accession number. **Parameters:** - `index_or_accession_number`: Integer index or accession number string **Returns:** `Filing` object **Example:** `# By index filing = filings.get(0) # By accession number filing = filings.get("0001234567-24-000001")` ### Subset Operations #### latest() `def latest(self, n: int = 1) -> Union[Filing, Filings]` Get most recent filing(s). **Parameters:** - `n` (int): Number of filings (default: 1) **Returns:** - Single `Filing` if n=1 - `Filings` collection if n>1 **Example:** `# Single latest filing latest_filing = filings.latest() # Latest 10 filings latest_10 = filings.latest(10)` #### head() `def head(self, n: int) -> Filings` Get first n filings. **Example:** `first_20 = filings.head(20)` #### tail() `def tail(self, n: int) -> Filings` Get last n filings. **Example:** `last_20 = filings.tail(20)` #### sample() `def sample(self, n: int) -> Filings` Get random sample of n filings. **Example:** `random_sample = filings.sample(10)` Filtering and Search -------------------- ### filter() `def filter( self, *, form: Optional[Union[str, List[str]]] = None, amendments: bool = None, filing_date: Optional[str] = None, date: Optional[str] = None, cik: Union[int, str, List[Union[int, str]]] = None, exchange: Union[str, List[str]] = None, ticker: Union[str, List[str]] = None, accession_number: Union[str, List[str]] = None ) -> Filings` Filter collection by multiple criteria. **Parameters:** - `form`: Form type(s) — e.g., "10-K", \["10-K", "10-Q"\] - `amendments`: Include/exclude amendments - `filing_date` / `date`: Date filter (YYYY-MM-DD or range) - `cik`: Central Index Key(s) - `exchange`: Exchange(s) — "NASDAQ", "NYSE", "CBOE", "OTC" - `ticker`: Stock ticker(s) - `accession_number`: Accession number(s) **Returns:** Filtered `Filings` collection **Examples:** #### Filter by form `# Single form annual = filings.filter(form="10-K") # Multiple forms financial = filings.filter(form=["10-K", "10-Q"]) # Exclude amendments original_only = filings.filter(form="10-K", amendments=False)` #### Filter by date `# Specific date jan_1 = filings.filter(date="2024-01-01") # Date range q1 = filings.filter(date="2024-01-01:2024-03-31") # From date onwards recent = filings.filter(date="2024-01-01:") # Up to date older = filings.filter(date=":2023-12-31")` #### Filter by company `# By ticker apple = filings.filter(ticker="AAPL") # By CIK apple = filings.filter(cik=320193) apple = filings.filter(cik="0000320193") # Multiple companies faang = filings.filter(ticker=["AAPL", "MSFT", "GOOGL"])` #### Filter by exchange `# Single exchange nasdaq = filings.filter(exchange="NASDAQ") # Multiple exchanges major = filings.filter(exchange=["NASDAQ", "NYSE"])` #### Chain filters `result = (filings .filter(form="10-K") .filter(exchange="NASDAQ") .filter(date="2024-01-01:") .latest(50))` ### find() `def find(self, company_search_str: str) -> Filings` Search for filings by company name. **Parameters:** - `company_search_str` (str): Company name search string **Returns:** Matching `Filings` collection **Example:** `# Find technology companies tech = filings.find("Technology") # Find specific company apple = filings.find("Apple")` Navigation and Pagination ------------------------- ### current() `def current(self) -> Filings` Get current page of filings. **Returns:** Current `Filings` page ### next() `def next(self) -> Optional[Filings]` Navigate to next page. **Returns:** Next page `Filings` or None **Example:** `next_page = filings.next() if next_page: print(f"Next page has {len(next_page)} filings")` ### previous() `def previous(self) -> Optional[Filings]` Navigate to previous page. **Returns:** Previous page `Filings` or None **Example:** `prev_page = filings.previous() if prev_page: print(f"Previous page has {len(prev_page)} filings")` Data Export and Persistence --------------------------- ### to\_pandas() `def to_pandas(self, *columns: str) -> pd.DataFrame` Convert to pandas DataFrame. **Parameters:** - `*columns`: Specific columns to include (optional) **Returns:** DataFrame with filing data **Example:** `# All columns df = filings.to_pandas() print(df.columns.tolist()) # Specific columns df = filings.to_pandas('form', 'company', 'filing_date', 'cik') print(df.head())` ### save\_parquet() / save() `def save_parquet(self, location: str) def save(self, location: str) # Alias` Save collection as Parquet file. **Parameters:** - `location` (str): File path **Example:** `filings.save_parquet("my_filings.parquet") # Load later import pandas as pd df = pd.read_parquet("my_filings.parquet")` ### to\_dict() `def to_dict(self, max_rows: int = 1000) -> Dict[str, Any]` Convert to dictionary. **Parameters:** - `max_rows` (int): Maximum rows (default: 1000) **Returns:** Dictionary representation ### to\_context() `def to_context(self, detail: str) -> str` Generate context string for LLM/AI use. **Parameters:** - `detail` (str): Level of detail **Returns:** Context string ### download() `def download( self, data_directory: Optional[str] = None, compress: bool = True, compression_level: int = 6, upload_to_cloud: bool = False, disable_progress: bool = False )` Download all filings in collection to local storage. **Parameters:** - `data_directory`: Download directory (defaults to Edgar data directory) - `compress`: Compress files (default: True) - `compression_level`: gzip level 1-9 (default: 6) - `upload_to_cloud`: Upload to cloud after download - `disable_progress`: Disable progress display **Example:** `# Download with defaults filings.download() # Custom directory filings.download(data_directory="./raw_data/", compress=False)` Common Recipes -------------- ### Get latest 10-K filings from major exchanges `from edgar import get_filings filings = get_filings(2024, 1, form="10-K") major_exchange = filings.filter(exchange=["NASDAQ", "NYSE"]) latest_20 = major_exchange.latest(20) print(f"Found {len(latest_20)} recent 10-K filings") for filing in latest_20: print(f"{filing.company}: {filing.filing_date}")` ### Analyze filing trends `from edgar import get_filings import pandas as pd filings = get_filings(2023, form=["10-K", "10-Q"]) df = filings.to_pandas() # Form distribution form_counts = df.groupby('form').size().sort_values(ascending=False) print("Form distribution:") print(form_counts) # Monthly trends df['filing_date'] = pd.to_datetime(df['filing_date']) monthly = df.groupby(df['filing_date'].dt.to_period('M')).size() print("\nMonthly filing counts:") print(monthly)` ### Batch process filings `from edgar import get_filings filings = get_filings(2024, 1, form="8-K") # Process in batches using pagination current_page = filings total_processed = 0 while current_page and not current_page.empty: for filing in current_page: # Process each filing text = filing.text() # ... analysis logic total_processed += len(current_page) print(f"Processed {total_processed} filings") # Move to next page current_page = current_page.next() print(f"Total processed: {total_processed}")` ### Filter and export for analysis `from edgar import get_filings # Get filings filings = get_filings(2023, form="10-K") # Filter to tech companies on NASDAQ nasdaq_tech = filings.filter(exchange="NASDAQ") # Export to DataFrame df = nasdaq_tech.to_pandas('company', 'filing_date', 'cik', 'accession_no') # Save for later analysis nasdaq_tech.save_parquet("nasdaq_tech_10k_2023.parquet")` ### Extract financial data from multiple filings `from edgar import get_filings filings = get_filings(2024, 1, form="10-K").head(50) results = [] for filing in filings: try: tenk = filing.obj() if tenk and tenk.financials: income = tenk.financials.income_statement # Extract data results.append({ 'company': filing.company, 'filing_date': filing.filing_date, 'has_financials': True }) except Exception as e: print(f"Error processing {filing.company}: {e}") print(f"Successfully processed {len(results)} filings")` Advanced Patterns ----------------- ### Chaining filters `# Build complex filters result = (get_filings(2024, 1) .filter(form=["10-K", "10-Q"]) .filter(exchange="NASDAQ") .filter(date="2024-01-01:2024-01-31") .latest(100))` ### Processing with pagination `def process_all_pages(filings): """Process all pages in a filings collection""" current = filings all_results = [] while current and not current.empty: # Process current page for filing in current: # Extract data all_results.append(filing.to_dict()) print(f"Processed page with {len(current)} filings") # Move to next page current = current.next() return all_results # Use it filings = get_filings(2023, form="10-K") results = process_all_pages(filings)` Performance Tips ---------------- 1. **Filter early** - Use `get_filings()` parameters instead of filtering later 2. **Limit results** - Use `.head(n)` or `.latest(n)` to avoid processing unnecessary filings 3. **Use pagination** - Process large datasets in pages with `.next()` 4. **Convert once** - Call `.to_pandas()` once and work with DataFrame **Efficient:** `filings = get_filings(2024, 1, form="10-K").head(100)` **Less efficient:** `filings = get_filings(2024, 1).filter(form="10-K").head(100)` Error Handling -------------- `from edgar import get_filings try: filings = get_filings(2024, 1, form="10-K") if filings.empty: print("No filings found") else: # Filter filtered = filings.filter(exchange="NASDAQ") if filtered.empty: print("No NASDAQ filings") else: # Process for filing in filtered: try: text = filing.text() except Exception as e: print(f"Error processing {filing.accession_no}: {e}") continue except Exception as e: print(f"Error: {e}")` Specialized Collections ----------------------- ### EntityFilings Company-specific filings with additional properties: `from edgar import Company company = Company("AAPL") entity_filings = company.get_filings() print(type(entity_filings)) # edgar.entity.filings.EntityFilings # Additional properties print(entity_filings.cik) print(entity_filings.company_name) # Methods return EntityFilings filtered = entity_filings.filter(form="10-K") # EntityFilings` ### CurrentFilings Real-time filings with enhanced pagination: `from edgar import get_current_filings current = get_current_filings() print(type(current)) # edgar._filings.CurrentFilings # Filter eightk = current.filter(form="8-K") # Navigate next_page = current.next()` See Also -------- * **[Filing API Reference](https://edgartools.readthedocs.io/en/stable/api/filing/) ** - Individual filing operations * **[Company API Reference](https://edgartools.readthedocs.io/en/stable/api/company/) ** - Company-specific filing access * **[Filtering Filings Guide](https://edgartools.readthedocs.io/en/stable/guides/filtering-filings/) ** - Advanced filtering techniques * **[Current Filings Guide](https://edgartools.readthedocs.io/en/stable/guides/current-filings/) ** - Real-time filing access * **[Search Filings Guide](https://edgartools.readthedocs.io/en/stable/guides/searching-filings/) ** - Finding specific filings Back to top --- # Filings - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/api/filings/#filings-api-reference-filter-and-navigate-sec-filing-collections) Filings API Reference — Filter and Navigate SEC Filing Collections ================================================================== The `Filings` class represents a collection of SEC filings with powerful filtering, navigation, and batch processing capabilities. **Quick example:** `from edgar import get_filings # Get Q1 2024 10-K filings filings = get_filings(2024, 1, form="10-K") print(f"Found {len(filings)} filings") # Filter to NASDAQ companies nasdaq = filings.filter(exchange="NASDAQ") # Get latest 10 latest = nasdaq.latest(10) # Export to DataFrame df = latest.to_pandas()` Getting Filings --------------- ### get\_filings() `def get_filings( year: int | List[int] = None, quarter: int | List[int] = None, form: str | List[str] = None, amendments: bool = True, filing_date: str = None ) -> Filings` **Parameters:** - `year` (int | List\[int\]): Calendar year(s) — NOT fiscal year - `quarter` (int | List\[int\]): Calendar quarter(s) 1-4 - `form` (str | List\[str\]): Form type(s) - `amendments` (bool): Include amendments (default: True) - `filing_date` (str): Date filter (YYYY-MM-DD or YYYY-MM-DD:YYYY-MM-DD) **Returns:** `Filings` collection **Important:** There is **no `limit` parameter**. Use `.head(n)` or `.latest(n)` on the result to limit. **Examples:** `from edgar import get_filings # Q1 2024 filings filings = get_filings(2024, 1) # All 2023 10-K filings filings = get_filings(2023, form="10-K") # Multiple years filings = get_filings([2022, 2023, 2024]) # Multiple quarters filings = get_filings(2024, [1, 2], form="10-Q") # Date range filings = get_filings(2024, 1, filing_date="2024-01-01:2024-01-31") # Multiple forms filings = get_filings(2024, 1, form=["10-K", "10-Q"]) # Exclude amendments filings = get_filings(2024, 1, form="10-K", amendments=False)` Core Properties --------------- ### Collection Information | Property | Type | Description | | --- | --- | --- | | `empty` | bool | Whether collection contains any filings | | `date_range` | Tuple\[str, str\] | (start\_date, end\_date) | | `start_date` | str | Earliest filing date | | `end_date` | str | Latest filing date | | `summary` | str | Summary description of collection | **Example:** `if filings.empty: print("No filings found") else: start, end = filings.date_range print(f"Filings from {start} to {end}") print(f"Total: {len(filings)}")` Collection Operations --------------------- ### Size and Access #### len() `len(filings) -> int` Number of filings in collection. **Example:** `print(f"Collection contains {len(filings)} filings")` #### Indexing `filings[index] -> Filing` Access individual filings by index. **Example:** `first = filings[0] last = filings[-1] # Slicing works but returns list, not Filings first_five = filings[:5] # List of Filing objects` #### Iteration `for filing in filings: ...` Iterate through collection. **Example:** `for filing in filings: print(f"{filing.form}: {filing.company} ({filing.filing_date})")` #### get() `def get(self, index_or_accession_number: Union[int, str]) -> Filing` Get filing by index or accession number. **Parameters:** - `index_or_accession_number`: Integer index or accession number string **Returns:** `Filing` object **Example:** `# By index filing = filings.get(0) # By accession number filing = filings.get("0001234567-24-000001")` ### Subset Operations #### latest() `def latest(self, n: int = 1) -> Union[Filing, Filings]` Get most recent filing(s). **Parameters:** - `n` (int): Number of filings (default: 1) **Returns:** - Single `Filing` if n=1 - `Filings` collection if n>1 **Example:** `# Single latest filing latest_filing = filings.latest() # Latest 10 filings latest_10 = filings.latest(10)` #### head() `def head(self, n: int) -> Filings` Get first n filings. **Example:** `first_20 = filings.head(20)` #### tail() `def tail(self, n: int) -> Filings` Get last n filings. **Example:** `last_20 = filings.tail(20)` #### sample() `def sample(self, n: int) -> Filings` Get random sample of n filings. **Example:** `random_sample = filings.sample(10)` Filtering and Search -------------------- ### filter() `def filter( self, *, form: Optional[Union[str, List[str]]] = None, amendments: bool = None, filing_date: Optional[str] = None, date: Optional[str] = None, cik: Union[int, str, List[Union[int, str]]] = None, exchange: Union[str, List[str]] = None, ticker: Union[str, List[str]] = None, accession_number: Union[str, List[str]] = None ) -> Filings` Filter collection by multiple criteria. **Parameters:** - `form`: Form type(s) — e.g., "10-K", \["10-K", "10-Q"\] - `amendments`: Include/exclude amendments - `filing_date` / `date`: Date filter (YYYY-MM-DD or range) - `cik`: Central Index Key(s) - `exchange`: Exchange(s) — "NASDAQ", "NYSE", "CBOE", "OTC" - `ticker`: Stock ticker(s) - `accession_number`: Accession number(s) **Returns:** Filtered `Filings` collection **Examples:** #### Filter by form `# Single form annual = filings.filter(form="10-K") # Multiple forms financial = filings.filter(form=["10-K", "10-Q"]) # Exclude amendments original_only = filings.filter(form="10-K", amendments=False)` #### Filter by date `# Specific date jan_1 = filings.filter(date="2024-01-01") # Date range q1 = filings.filter(date="2024-01-01:2024-03-31") # From date onwards recent = filings.filter(date="2024-01-01:") # Up to date older = filings.filter(date=":2023-12-31")` #### Filter by company `# By ticker apple = filings.filter(ticker="AAPL") # By CIK apple = filings.filter(cik=320193) apple = filings.filter(cik="0000320193") # Multiple companies faang = filings.filter(ticker=["AAPL", "MSFT", "GOOGL"])` #### Filter by exchange `# Single exchange nasdaq = filings.filter(exchange="NASDAQ") # Multiple exchanges major = filings.filter(exchange=["NASDAQ", "NYSE"])` #### Chain filters `result = (filings .filter(form="10-K") .filter(exchange="NASDAQ") .filter(date="2024-01-01:") .latest(50))` ### find() `def find(self, company_search_str: str) -> Filings` Search for filings by company name. **Parameters:** - `company_search_str` (str): Company name search string **Returns:** Matching `Filings` collection **Example:** `# Find technology companies tech = filings.find("Technology") # Find specific company apple = filings.find("Apple")` Navigation and Pagination ------------------------- ### current() `def current(self) -> Filings` Get current page of filings. **Returns:** Current `Filings` page ### next() `def next(self) -> Optional[Filings]` Navigate to next page. **Returns:** Next page `Filings` or None **Example:** `next_page = filings.next() if next_page: print(f"Next page has {len(next_page)} filings")` ### previous() `def previous(self) -> Optional[Filings]` Navigate to previous page. **Returns:** Previous page `Filings` or None **Example:** `prev_page = filings.previous() if prev_page: print(f"Previous page has {len(prev_page)} filings")` Data Export and Persistence --------------------------- ### to\_pandas() `def to_pandas(self, *columns: str) -> pd.DataFrame` Convert to pandas DataFrame. **Parameters:** - `*columns`: Specific columns to include (optional) **Returns:** DataFrame with filing data **Example:** `# All columns df = filings.to_pandas() print(df.columns.tolist()) # Specific columns df = filings.to_pandas('form', 'company', 'filing_date', 'cik') print(df.head())` ### save\_parquet() / save() `def save_parquet(self, location: str) def save(self, location: str) # Alias` Save collection as Parquet file. **Parameters:** - `location` (str): File path **Example:** `filings.save_parquet("my_filings.parquet") # Load later import pandas as pd df = pd.read_parquet("my_filings.parquet")` ### to\_dict() `def to_dict(self, max_rows: int = 1000) -> Dict[str, Any]` Convert to dictionary. **Parameters:** - `max_rows` (int): Maximum rows (default: 1000) **Returns:** Dictionary representation ### to\_context() `def to_context(self, detail: str) -> str` Generate context string for LLM/AI use. **Parameters:** - `detail` (str): Level of detail **Returns:** Context string ### download() `def download( self, data_directory: Optional[str] = None, compress: bool = True, compression_level: int = 6, upload_to_cloud: bool = False, disable_progress: bool = False )` Download all filings in collection to local storage. **Parameters:** - `data_directory`: Download directory (defaults to Edgar data directory) - `compress`: Compress files (default: True) - `compression_level`: gzip level 1-9 (default: 6) - `upload_to_cloud`: Upload to cloud after download - `disable_progress`: Disable progress display **Example:** `# Download with defaults filings.download() # Custom directory filings.download(data_directory="./raw_data/", compress=False)` Common Recipes -------------- ### Get latest 10-K filings from major exchanges `from edgar import get_filings filings = get_filings(2024, 1, form="10-K") major_exchange = filings.filter(exchange=["NASDAQ", "NYSE"]) latest_20 = major_exchange.latest(20) print(f"Found {len(latest_20)} recent 10-K filings") for filing in latest_20: print(f"{filing.company}: {filing.filing_date}")` ### Analyze filing trends `from edgar import get_filings import pandas as pd filings = get_filings(2023, form=["10-K", "10-Q"]) df = filings.to_pandas() # Form distribution form_counts = df.groupby('form').size().sort_values(ascending=False) print("Form distribution:") print(form_counts) # Monthly trends df['filing_date'] = pd.to_datetime(df['filing_date']) monthly = df.groupby(df['filing_date'].dt.to_period('M')).size() print("\nMonthly filing counts:") print(monthly)` ### Batch process filings `from edgar import get_filings filings = get_filings(2024, 1, form="8-K") # Process in batches using pagination current_page = filings total_processed = 0 while current_page and not current_page.empty: for filing in current_page: # Process each filing text = filing.text() # ... analysis logic total_processed += len(current_page) print(f"Processed {total_processed} filings") # Move to next page current_page = current_page.next() print(f"Total processed: {total_processed}")` ### Filter and export for analysis `from edgar import get_filings # Get filings filings = get_filings(2023, form="10-K") # Filter to tech companies on NASDAQ nasdaq_tech = filings.filter(exchange="NASDAQ") # Export to DataFrame df = nasdaq_tech.to_pandas('company', 'filing_date', 'cik', 'accession_no') # Save for later analysis nasdaq_tech.save_parquet("nasdaq_tech_10k_2023.parquet")` ### Extract financial data from multiple filings `from edgar import get_filings filings = get_filings(2024, 1, form="10-K").head(50) results = [] for filing in filings: try: tenk = filing.obj() if tenk and tenk.financials: income = tenk.financials.income_statement # Extract data results.append({ 'company': filing.company, 'filing_date': filing.filing_date, 'has_financials': True }) except Exception as e: print(f"Error processing {filing.company}: {e}") print(f"Successfully processed {len(results)} filings")` Advanced Patterns ----------------- ### Chaining filters `# Build complex filters result = (get_filings(2024, 1) .filter(form=["10-K", "10-Q"]) .filter(exchange="NASDAQ") .filter(date="2024-01-01:2024-01-31") .latest(100))` ### Processing with pagination `def process_all_pages(filings): """Process all pages in a filings collection""" current = filings all_results = [] while current and not current.empty: # Process current page for filing in current: # Extract data all_results.append(filing.to_dict()) print(f"Processed page with {len(current)} filings") # Move to next page current = current.next() return all_results # Use it filings = get_filings(2023, form="10-K") results = process_all_pages(filings)` Performance Tips ---------------- 1. **Filter early** - Use `get_filings()` parameters instead of filtering later 2. **Limit results** - Use `.head(n)` or `.latest(n)` to avoid processing unnecessary filings 3. **Use pagination** - Process large datasets in pages with `.next()` 4. **Convert once** - Call `.to_pandas()` once and work with DataFrame **Efficient:** `filings = get_filings(2024, 1, form="10-K").head(100)` **Less efficient:** `filings = get_filings(2024, 1).filter(form="10-K").head(100)` Error Handling -------------- `from edgar import get_filings try: filings = get_filings(2024, 1, form="10-K") if filings.empty: print("No filings found") else: # Filter filtered = filings.filter(exchange="NASDAQ") if filtered.empty: print("No NASDAQ filings") else: # Process for filing in filtered: try: text = filing.text() except Exception as e: print(f"Error processing {filing.accession_no}: {e}") continue except Exception as e: print(f"Error: {e}")` Specialized Collections ----------------------- ### EntityFilings Company-specific filings with additional properties: `from edgar import Company company = Company("AAPL") entity_filings = company.get_filings() print(type(entity_filings)) # edgar.entity.filings.EntityFilings # Additional properties print(entity_filings.cik) print(entity_filings.company_name) # Methods return EntityFilings filtered = entity_filings.filter(form="10-K") # EntityFilings` ### CurrentFilings Real-time filings with enhanced pagination: `from edgar import get_current_filings current = get_current_filings() print(type(current)) # edgar._filings.CurrentFilings # Filter eightk = current.filter(form="8-K") # Navigate next_page = current.next()` See Also -------- * **[Filing API Reference](https://edgartools.readthedocs.io/en/latest/api/filing/) ** - Individual filing operations * **[Company API Reference](https://edgartools.readthedocs.io/en/latest/api/company/) ** - Company-specific filing access * **[Filtering Filings Guide](https://edgartools.readthedocs.io/en/latest/guides/filtering-filings/) ** - Advanced filtering techniques * **[Current Filings Guide](https://edgartools.readthedocs.io/en/latest/guides/current-filings/) ** - Real-time filing access * **[Search Filings Guide](https://edgartools.readthedocs.io/en/latest/guides/searching-filings/) ** - Finding specific filings Back to top --- # Standardization - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/xbrl/concepts/standardization/#xbrl-standardization-concepts-reference) XBRL Standardization Concepts Reference ======================================= This document describes the 95 standard concepts used by EdgarTools to normalize XBRL financial data across different companies. These concepts enable cross-company comparison by mapping the ~18,000 possible SEC GAAP taxonomy tags to a consistent set of standardized line items. How Standardization Works ------------------------- **Labels are always preserved** - the company's original presentation is shown exactly as filed. Standardization adds a `standard_concept` column to DataFrames, mapping each line item to one of 95 standard categories: `# Get a statement DataFrame df = statement.to_dataframe() # Labels show original company presentation # standard_concept maps to standard categories for analysis print(df[['label', 'standard_concept']].head()) # label standard_concept # 0 Cash and cash items CashAndMarketableSecurities # 1 Trade receivables, net TradeReceivables # 2 Prepaid expenses OtherNonOperatingCurrentAssets # Aggregate by standard concept for cross-company comparison standardized = df.groupby('standard_concept')[['2024-09-30']].sum()` Overview -------- | Metric | Value | | --- | --- | | **Total Standard Concepts** | 95 | | **XBRL Tags Mapped** | 2,067 | | **Coverage** | ~95% of common financial statement tags | | **Source** | mpreiss9's production taxonomy (390+ companies) | Architecture ------------ `XBRL Tag (e.g., AccountsPayableCurrent) ↓ gaap_mappings.json (2,067 mappings) Standard Concept (e.g., TradePayables) ↓ display_names.json (95 mappings) Display Name (e.g., "Accounts Payable")` Industry Context ---------------- The number of standardized concepts varies by provider: | Provider | Line Items | Notes | | --- | --- | --- | | EdgarTools | 95 | Production-tested on 390+ companies | | Capital IQ | ~100-150 | Varies by data package | | Bloomberg | ~80-120 | Core financial line items | | Refinitiv | ~100-200 | Standardized fundamentals | | Morningstar | ~80-100 | Balance/Income/Cash Flow | * * * Standard Concepts by Statement ------------------------------ ### Balance Sheet - Current Assets | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `CashAndMarketableSecurities` | Cash and Cash Equivalents | 61 | `AssetBackedSecuritiesAtCarryingValue`, `AvailableForSaleDebtSecuritiesAmortizedCostBasis`, `AvailableForSaleSecurities`, ... | | `TradeReceivables` | Accounts Receivable | 34 | `AccountsAndNotesReceivableNet`, `AccountsAndOtherReceivablesNetCurrent`, `AccountsNotesAndLoansReceivableNetCurrent`, ... | | `Inventories` | Inventory | 66 | `AgriculturalRelatedInventory`, `AgriculturalRelatedInventoryFeedAndSupplies`, `AgriculturalRelatedInventoryGrowingCrops`, ... | | `DeferredTaxCurrentAssets` | Deferred Tax Assets, Current | 44 | `DeferredIncomeTaxesAndOtherAssetsCurrent`, `DeferredIncomeTaxesAndOtherTaxReceivableCurrent`, `DeferredTaxAssetsDeferredIncome`, ... | | `OtherOperatingCurrentAssets` | Other Current Assets | 50 | `AdvanceRoyaltiesCurrent`, `AdvancesOnInventoryPurchases`, `AmountOfDeferredCostsRelatedToLongTermContracts`, ... | | `OtherNonOperatingCurrentAssets` | Other Non-Operating Current Assets | 131 | `AccountsReceivableRelatedParties`, `AccountsReceivableRelatedPartiesCurrent`, `AllowanceForDoubtfulOtherReceivablesCurrent`, ... | | `RetirementRelatedCurrentAssets` | Retirement Related Assets, Current | 1 | `DefinedBenefitPlanCurrentAssets` | | `CurrentAssetsTotal` | Total Current Assets | 1 | `AssetsCurrent` | ### Balance Sheet - Non-Current Assets | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `PlantPropertyEquipmentNet` | Property, Plant and Equipment | 53 | `AccumulatedDepreciationDepletionAndAmortizationPropertyPlantAndEquipment`, `AcquisitionCostsCumulative`, `BuildingsAndImprovementsGross`, ... | | `Goodwill` | Goodwill | 10 | `Goodwill`, `GoodwillGross`, `GoodwillImpairedAccumulatedImpairmentLoss`, ... | | `IntangibleAssets` | Intangible Assets | 17 | `BusinessCombinationRecognizedIdentifiableAssetsAcquiredAndLiabilitiesAssumedIntangibles`, `FiniteLivedCustomerListsGross`, `FiniteLivedCustomerRelationshipsGross`, ... | | `LongtermInvestments` | Long-Term Investments | 60 | `AdvancesToAffiliate`, `AuctionRateSecuritiesNoncurrent`, `AvailableForSaleSecuritiesDebtMaturitiesAfterFiveThroughTenYearsFairValue`, ... | | `DeferredTaxNoncurrentAssets` | Deferred Tax Assets, Non-Current | 45 | `DeferredIncomeTaxAssetsNet`, `DeferredIncomeTaxesAndOtherAssetsNoncurrent`, `DeferredTaxAssetsCapitalLossCarryforwards`, ... | | `OtherOperatingNonCurrentAssets` | Other Non-Current Assets | 49 | `AccountsReceivableExcludingAccruedInterestAfterAllowanceForCreditLossNoncurrent`, `AccountsReceivableGross`, `AccountsReceivableGrossNoncurrent`, ... | | `OtherNonOperatingNonCurrentAssets` | Other Non-Operating Non-Current Assets | 140 | `AccountsReceivableRelatedParties`, `AccountsReceivableRelatedPartiesNoncurrent`, `AccruedFeesAndOtherRevenueReceivable`, ... | | `OperatingLeaseRightOfUseAsset` | Operating Lease Right-of-Use Asset | 1 | `OperatingLeaseRightOfUseAsset` | | `RetirementRelatedNonCurrentAssets` | Retirement Related Assets, Non-Current | 4 | `AssetRetirementObligationLegallyRestrictedAssetsFairValue`, `DeferredCompensationPlanAssets`, `DefinedBenefitPlanAmountsRecognizedInBalanceSheet`, ... | | `NonCurrentAssetsTotal` | Total Non-Current Assets | 2 | `AssetsNoncurrent`, `NoncurrentAssets` | | `Assets` | Total Assets | 2 | `Assets`, `AssetsNet` | ### Balance Sheet - Current Liabilities | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `TradePayables` | Accounts Payable | 27 | `AccountsPayableAndAccruedLiabilitiesCurrent`, `AccountsPayableAndAccruedLiabilitiesCurrentAndNoncurrent`, `AccountsPayableAndOtherAccruedLiabilities`, ... | | `ShortTermDebt` | Short-Term Debt | 62 | `BankLoans`, `BankOverdrafts`, `BorrowingsUnderGuaranteedInvestmentAgreements`, ... | | `DeferredTaxCurrentLiabilities` | Deferred Tax Liabilities, Current | 20 | `DeferredIncomeTaxLiabilities`, `DeferredIncomeTaxLiabilitiesNet`, `DeferredTaxAssetsLiabilitiesNetCurrent`, ... | | `TaxesPayable` | Taxes Payable | 7 | `AccrualForTaxesOtherThanIncomeTaxesCurrent`, `AccruedIncomeTaxes`, `AccruedIncomeTaxesCurrent`, ... | | `DividendsPayable` | Dividends Payable | 2 | `DividendsPayableCurrent`, `DividendsPayableCurrentAndNoncurrent` | | `OtherOperatingCurrentLiabilities` | Other Current Liabilities | 68 | `AccrualForEnvironmentalLossContingencies`, `AccruedAdvertisingCurrent`, `AccruedAdvertisingCurrentAndNoncurrent`, ... | | `OtherNonOperatingCurrentLiabilities` | Other Non-Operating Current Liabilities | 82 | `AccountsPayableOtherCurrentAndNoncurrent`, `AccrualForTaxesOtherThanIncomeTaxesCurrentAndNoncurrent`, `AccruedCappingClosurePostClosureAndEnvironmentalCosts`, ... | | `RetirementRelatedCurrentLiabilities` | Retirement Related Liabilities, Current | 18 | `DeferredCompensationCashBasedArrangementsLiabilityCurrent`, `DeferredCompensationLiabilityCurrent`, `DeferredCompensationLiabilityCurrentAndNoncurrent`, ... | | `OperatingLeaseCurrentDebtEquivalent` | Operating Lease Liability, Current | 2 | `OperatingLeaseLiability`, `OperatingLeaseLiabilityCurrent` | | `CurrentLiabilitiesTotal` | Total Current Liabilities | 1 | `LiabilitiesCurrent` | ### Balance Sheet - Non-Current Liabilities | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `LongTermDebt` | Long-Term Debt | 71 | `CapitalLeaseObligations`, `CapitalLeaseObligationsNoncurrent`, `CommercialPaperNoncurrent`, ... | | `DeferredTaxNonCurrentLiabilities` | Deferred Tax Liabilities, Non-Current | 35 | `AccruedIncomeTaxes`, `AccruedIncomeTaxesNoncurrent`, `AccumulatedDeferredInvestmentTaxCredit`, ... | | `OtherOperatingNonCurrentLiabilities` | Other Non-Current Liabilities | 31 | `AccountsPayableAndAccruedLiabilitiesCurrentAndNoncurrent`, `AccountsPayableAndAccruedLiabilitiesNoncurrent`, `AccountsPayableAndOtherAccruedLiabilities`, ... | | `OtherNonOperatingNonCurrentLiabilities` | Other Non-Operating Non-Current Liabilities | 81 | `AccountsPayableOtherCurrentAndNoncurrent`, `AccrualForTaxesOtherThanIncomeTaxesCurrentAndNoncurrent`, `AccruedEmployeeBenefitsCurrentAndNoncurrent`, ... | | `RetirementRelatedNonCurrentLiabilities` | Retirement Related Liabilities, Non-Current | 18 | `AssetRetirementObligation`, `DeferredCompensationLiabilityClassifiedNoncurrent`, `DeferredCompensationLiabilityCurrentAndNoncurrent`, ... | | `OperatingLeaseNonCurrentDebtEquivalent` | Operating Lease Liability, Non-Current | 3 | `OperatingLeaseLiability`, `OperatingLeaseLiabilityNoncurrent`, `OperatingLeaseLiabilityStatementOfFinancialPositionExtensibleList` | | `OngoingOperatingProvisions(WarrantiesEtc)` | Warranty and Other Provisions | 22 | `ContractWithCustomerRefundLiability`, `ContractWithCustomerRefundLiabilityNoncurrent`, `CustomerAdvancesAndDeposits`, ... | | `DefinteLivedOperatingProvisions(DecommissioningEtc)` | Asset Retirement Obligations | 10 | `AccruedCappingClosurePostClosureAndEnvironmentalCosts`, `AccruedCappingClosurePostClosureAndEnvironmentalCostsNoncurrent`, `AssetRetirementObligationsNoncurrent`, ... | | `RestructuringProvisions` | Restructuring Provisions | 2 | `RestructuringReserve`, `RestructuringReserveNoncurrent` | | `NonCurrentLiabilitiesTotal` | Total Non-Current Liabilities | 1 | `LiabilitiesNoncurrent` | | `Liabilities` | Total Liabilities | 1 | `Liabilities` | ### Balance Sheet - Equity | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `CommonEquity` | Total Stockholders' Equity | 50 | `AccumulatedOtherComprehensiveIncomeLossAvailableForSaleSecuritiesAdjustmentNetOfTax`, `AccumulatedOtherComprehensiveIncomeLossCumulativeChangesInNetGainLossFromCashFlowHedgesEffectNetOfTax`, `AccumulatedOtherComprehensiveIncomeLossDefinedBenefitPensionAndOtherPostretirementPlansNetOfTax`, ... | | `PreferredStock` | Preferred Stock | 6 | `AdditionalPaidInCapitalPreferredStock`, `PreferredStockRedemptionAmount`, `PreferredStockSharesSubscribedButUnissuedSubscriptionsReceivable`, ... | | `TreasuryShares` | Treasury Stock | 2 | `TreasuryStockCommonShares`, `TreasuryStockShares` | | `MinorityInterestBalance` | Noncontrolling Interest | 10 | `MembersEquityAttributableToNoncontrollingInterest`, `MinorityInterest`, `MinorityInterestInJointVentures`, ... | | `TemporaryAndMezzanineFinancing` | Temporary Equity | 11 | `RedeemableNoncontrollingInterestEquityCarryingAmount`, `RedeemableNoncontrollingInterestEquityCommonCarryingAmount`, `RedeemableNoncontrollingInterestEquityCommonFairValue`, ... | | `AllEquityBalance` | Total Equity | 1 | `StockholdersEquity` | | `AllEquityBalanceIncludingMinorityInterest` | Total Equity Including Noncontrolling Interest | 4 | `AociIncludingPortionAttributableToNoncontrollingInterestTax`, `DefinedBenefitPlanAccumulatedOtherComprehensiveIncomeBeforeTax`, `LimitedLiabilityCompanyLlcMembersEquityIncludingPortionAttributableToNoncontrollingInterest`, ... | | `LiabilitiesAndEquity` | Total Liabilities and Equity | 1 | `LiabilitiesAndStockholdersEquity` | ### Income Statement - Revenue & Gross Profit | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `Revenue` | Revenue | 139 | `AdmissionsRevenue`, `AdvertisingRevenue`, `BinderSalesRevenue`, ... | | `CostOfGoodsAndServicesSold` | Cost of Revenue | 142 | `AdvertisingRevenueCost`, `AffiliateCosts`, `AircraftRentalAndLandingFees`, ... | | `GrossProfit` | Gross Profit | 1 | `GrossProfit` | ### Income Statement - Operating Expenses | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `ResearchAndDevelopementExpenses` | Research and Development Expense | 6 | `ExplorationExpense`, `ResearchAndDevelopmentAssetAcquiredOtherThanThroughBusinessCombinationWrittenOff`, `ResearchAndDevelopmentExpense`, ... | | `SellingGeneralAndAdminExpenses` | Selling, General and Administrative Expense | 16 | `CommunicationsAndInformationTechnology`, `GeneralAndAdministrativeExpense`, `GeneralInsuranceExpense`, ... | | `MarketingExpenses` | Marketing Expense | 4 | `AdvertisingExpense`, `CooperativeAdvertisingExpense`, `MarketingAndAdvertisingExpense`, ... | | `DepreciationExpense` | Depreciation Expense | 13 | `CapitalizedComputerSoftwareAmortization`, `CapitalizedComputerSoftwareImpairments`, `CostDepreciationAmortizationAndDepletion`, ... | | `AmortizationOfIntangibles` | Amortization of Intangibles | 4 | `AmortizationOfIntangibleAssets`, `ImpairmentOfIntangibleAssetsExcludingGoodwill`, `ImpairmentOfIntangibleAssetsFinitelived`, ... | | `OtherOperatingExpense` | Other Operating Expense | 45 | `AccretionExpense`, `AcquisitionCosts`, `AllocatedShareBasedCompensationExpense`, ... | | `RestructuringExpenseBenefit` | Restructuring Expense | 32 | `AmortizationOfAcquisitionCosts`, `BusinessCombinationAcquisitionRelatedCosts`, `BusinessCombinationIntegrationRelatedCosts`, ... | | `GoodwillWriteoffs` | Goodwill Impairment | 9 | `AdjustmentForAmortization`, `AssetImpairmentCharges`, `CostOfGoodsAndServicesSoldAmortization`, ... | | `CostsSubtotal` | Total Costs and Expenses | 5 | `BenefitsLossesAndExpenses`, `CostsAndExpenses`, `EmployeeBenefitsAndShareBasedCompensation`, ... | | `OperatingIncomeLoss` | Operating Income | 1 | `OperatingIncomeLoss` | ### Income Statement - Non-Operating & Tax | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `InterestExpense` | Interest Expense | 38 | `AmortizationOfDebtDiscountPremium`, `AmortizationOfDeferredHedgeGains`, `AmortizationOfFinancingCosts`, ... | | `InterestIncome` | Interest Income | 20 | `InterestAndDividendIncomeOperating`, `InterestAndOtherIncome`, `InterestIncomeExpenseAfterProvisionForLoanLoss`, ... | | `NonoperatingIncomeExpense` | Non-Operating Income (Expense) | 199 | `AccretionAmortizationOfDiscountsAndPremiumsInvestments`, `AccretionExpenseIncludingAssetRetirementObligations`, `AvailableForSaleSecuritiesGrossRealizedGainLossNet`, ... | | `SpecialItemsIncomeExpense(Pretax)` | Special Items | 2 | `UnusualOrInfrequentItemInsuranceProceeds`, `UnusualOrInfrequentItemNetOfInsuranceProceeds` | | `PretaxIncomeLoss` | Income Before Tax | 1 | `IncomeLossFromContinuingOperationsBeforeIncomeTaxesExtraordinaryItemsNoncontrollingInterest` | | `IncomeTaxes` | Income Tax Expense | 78 | `AdjustmentsToAdditionalPaidInCapitalTaxEffectFromShareBasedCompensation`, `CurrentFederalStateAndLocalTaxExpenseBenefit`, `CurrentFederalTaxExpenseBenefit`, ... | | `IncomeLossContinuingOperations` | Income from Continuing Operations | 1 | `IncomeLossFromContinuingOperations` | | `ExtraordinaryItemsIncomeExpense(PostTax)` | Extraordinary Items | 35 | `DiscontinuedOperationAmountOfAdjustmentToPriorPeriodGainLossOnDisposalBeforeIncomeTax`, `DiscontinuedOperationAmountOfAdjustmentToPriorPeriodGainLossOnDisposalNetOfTax`, `DiscontinuedOperationAmountOfOtherIncomeLossFromDispositionOfDiscontinuedOperationNetOfTax`, ... | | `MinorityInterestIncomeExpense` | Net Income Attributable to Noncontrolling Interest | 10 | `ComprehensiveIncomeNetOfTaxAttributableToNoncontrollingInterest`, `EquityMethodInvestmentOtherThanTemporaryImpairment`, `IncomeLossFromContinuingOperationsAttributableToNoncontrollingEntity`, ... | | `NetIncome` | Net Income | 2 | `IncomeLossAttributableToParent`, `NetIncomeLoss` | | `NetIncomeToCommonShareholders` | Net Income to Common Shareholders | 3 | `NetIncomeLossAvailableToCommonStockholdersBasic`, `NetIncomeLossFromContinuingOperationsAvailableToCommonShareholdersBasic`, `ParticipatingSecuritiesDistributedAndUndistributedEarnings` | | `PreferredDividendExpense` | Preferred Dividends | 13 | `DividendsPreferredStock`, `DividendsPreferredStockCash`, `DividendsPreferredStockStock`, ... | | `ProfitLoss` | Profit or Loss | 1 | `ProfitLoss` | ### Cash Flow & Capital | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `CapitalExpenses` | Capital Expenditures | 4 | `PaymentsToAcquireOtherProductiveAssets`, `PaymentsToAcquireOtherPropertyPlantAndEquipment`, `PaymentsToAcquireProductiveAssets`, ... | | `CommonDividendsPaid` | Dividends Paid | 4 | `Dividends`, `DividendsCash`, `DividendsCommonStock`, ... | | `EquityExpenseIncome(BuybackIssued)` | Stock Repurchases (Issuances) | 4 | `PaymentsForRepurchaseOfCommonStock`, `ProceedsFromIssuanceOfCommonStock`, `ProceedsFromSaleOfTreasuryStock`, ... | ### Per Share Data | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `CommonDividendsPerShare` | Dividends Per Share | 3 | `CommonStockDividendsPerShareCashPaid`, `CommonStockDividendsPerShareDeclared`, `DividendsPayableAmountPerShare` | | `SharesAverage` | Weighted Average Shares Outstanding | 2 | `WeightedAverageBasicSharesOutstandingProForma`, `WeightedAverageNumberOfSharesOutstandingBasic` | | `SharesDilutionAdjustment` | Dilution Adjustment | 2 | `IncrementalCommonSharesAttributableToShareBasedPaymentArrangements`, `WeightedAverageNumberDilutedSharesOutstandingAdjustment` | | `SharesFullyDilutedAverage` | Weighted Average Shares Outstanding, Diluted | 2 | `WeightedAverageNumberOfDilutedSharesOutstanding`, `WeightedAverageNumberOfShareOutstandingBasicAndDiluted` | | `SharesIssued` | Shares Issued | 2 | `CommonStockSharesIssued`, `SharesIssued` | | `SharesYearEnd` | Shares Outstanding | 3 | `CommonStockSharesOutstanding`, `EntityCommonStockSharesOutstanding`, `SharesOutstanding` | ### Operating Lease Commitments | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `OperatingLeaseCommitmentYear1` | Operating Lease Commitment, Year 1 | 1 | `OperatingLeasesFutureMinimumPaymentsDueCurrent` | | `OperatingLeaseCommitmentYear2` | Operating Lease Commitment, Year 2 | 1 | `OperatingLeasesFutureMinimumPaymentsDueInTwoYears` | | `OperatingLeaseCommitmentYear3` | Operating Lease Commitment, Year 3 | 1 | `OperatingLeasesFutureMinimumPaymentsDueInThreeYears` | | `OperatingLeaseCommitmentYear4` | Operating Lease Commitment, Year 4 | 1 | `OperatingLeasesFutureMinimumPaymentsDueInFourYears` | | `OperatingLeaseCommitmentYear5` | Operating Lease Commitment, Year 5 | 1 | `OperatingLeasesFutureMinimumPaymentsDueInFiveYears` | | `OperatingLeaseCommitmentAfterYear5` | Operating Lease Commitment, Thereafter | 1 | `OperatingLeasesFutureMinimumPaymentsDueThereafter` | ### Intangible Amortization Forecast | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `ForecastedIntangibleAmortizationYear1` | Forecasted Amortization, Year 1 | 3 | `FiniteLivedIntangibleAssetsAmortizationExpenseNextRollingTwelveMonths`, `FiniteLivedIntangibleAssetsAmortizationExpenseNextTwelveMonths`, `FiniteLivedIntangibleAssetsAmortizationExpenseRemainderOfFiscalYear` | | `ForecastedIntangibleAmortizationYear2` | Forecasted Amortization, Year 2 | 2 | `FiniteLivedIntangibleAssetsAmortizationExpenseRollingYearTwo`, `FiniteLivedIntangibleAssetsAmortizationExpenseYearTwo` | | `ForecastedIntangibleAmortizationYear3` | Forecasted Amortization, Year 3 | 2 | `FiniteLivedIntangibleAssetsAmortizationExpenseRollingYearThree`, `FiniteLivedIntangibleAssetsAmortizationExpenseYearThree` | | `ForecastedIntangibleAmortizationYear4` | Forecasted Amortization, Year 4 | 1 | `FiniteLivedIntangibleAssetsAmortizationExpenseYearFour` | | `ForecastedIntangibleAmortizationYear5` | Forecasted Amortization, Year 5 | 1 | `FiniteLivedIntangibleAssetsAmortizationExpenseYearFive` | | `ForecastedIntangibleAmortizationAfterYear5` | Forecasted Amortization, Thereafter | 2 | `FiniteLivedIntangibleAssetsAmortizationExpenseAfterYearFive`, `FiniteLivedIntangibleAssetsAmortizationExpenseRollingAfterYearFive` | * * * Concept Details --------------- ### Balance Sheet Concepts #### Current vs Non-Current Classification The taxonomy distinguishes between current (due within 1 year) and non-current items: * **Current Assets**: `CashAndMarketableSecurities`, `TradeReceivables`, `Inventories`, etc. * **Non-Current Assets**: `PlantPropertyEquipmentNet`, `Goodwill`, `LongtermInvestments`, etc. * **Current Liabilities**: `TradePayables`, `ShortTermDebt`, etc. * **Non-Current Liabilities**: `LongTermDebt`, `DeferredTaxNonCurrentLiabilities`, etc. #### Operating vs Non-Operating Classification Items are also classified by their relationship to core operations: * **Operating**: Related to primary business activities (e.g., `OtherOperatingCurrentAssets`) * **Non-Operating**: Related to financing/investing activities (e.g., `OtherNonOperatingCurrentAssets`) ### Income Statement Concepts #### Revenue Recognition All revenue-related XBRL tags (139 variations) map to the single `Revenue` concept. This includes: - `RevenueFromContractWithCustomerExcludingAssessedTax` - `Revenues` - `SalesRevenueNet` - `SalesRevenueGoodsNet` - And 135 more industry-specific variations #### Cost of Revenue Cost tags (142 variations) map to `CostOfGoodsAndServicesSold`: - `CostOfRevenue` - `CostOfGoodsAndServicesSold` - `CostOfGoodsSold` - `CostOfServices` ### Operating Lease Accounting (ASC 842) Following the ASC 842 lease accounting standard, the taxonomy includes: * **Right-of-Use Asset**: `OperatingLeaseRightOfUseAsset` * **Current Liability**: `OperatingLeaseCurrentDebtEquivalent` * **Non-Current Liability**: `OperatingLeaseNonCurrentDebtEquivalent` * **Future Commitments**: Years 1-5 and thereafter ### Provisions and Reserves The taxonomy distinguishes between: * **Ongoing Provisions**: `OngoingOperatingProvisions(WarrantiesEtc)` - recurring obligations * **Definite-Lived Provisions**: `DefinteLivedOperatingProvisions(DecommissioningEtc)` - asset retirement * **Restructuring**: `RestructuringProvisions` - one-time reorganization costs * * * Ambiguous Tags -------------- Some XBRL tags can map to multiple concepts depending on context. These are flagged as "ambiguous" and require context-aware resolution (Phase 4 of implementation). **Total Ambiguous Tags**: 215 (9.2% of mapped tags) ### Common Ambiguity Types 1. **Current/Non-Current Ambiguity** (202 tags) 2. Example: `AccountsPayableCurrentAndNoncurrent` → `TradePayables` OR `OtherOperatingNonCurrentLiabilities` 3. Resolution: Based on balance sheet section placement 4. **Asset/Liability Ambiguity** (12 tags) 5. Example: `DeferredTaxAssetsLiabilitiesNet` → `DeferredTaxNoncurrentAssets` OR `DeferredTaxNonCurrentLiabilities` 6. Resolution: Based on sign (positive = asset, negative = liability) 7. **Operating/Non-Operating Ambiguity** 8. Example: `OtherAssetsNoncurrent` → `OtherOperatingNonCurrentAssets` OR `OtherNonOperatingNonCurrentAssets` 9. Resolution: Based on statement section context * * * Excluded Tags (DropThisItem) ---------------------------- 276 XBRL tags are explicitly excluded from standardization because they: - Confuse financial analysis (EPS details, pro-forma metrics) - Are redundant with other tags - Don't map cleanly to standard concepts **Examples of excluded tags:** - `AcceleratedShareRepurchasesFinalPricePaidPerShare` - `BasicEarningsPerShareProForma` - `BusinessAcquisitionProFormaEarningsPerShareBasic` - Various per-share calculation details * * * Deprecated Tags --------------- 410 XBRL tags are marked as deprecated by the SEC with the year of deprecation. The mapping still works for historical filings, but these tags should not appear in recent filings. **Example:** - `Revenues` (deprecated 2018) → Still maps to `Revenue` - `AccountsPayableRelatedPartiesCurrent` (deprecated 2023) → Still maps to `TradePayables` * * * Python API (v5.9.0+) -------------------- EdgarTools provides several APIs for working with standardized concepts: ### Module-Level Singletons (Recommended) For best performance, use the module-level singletons which load mappings once per session: `from edgar.xbrl.standardization import ( get_default_mapper, get_default_store, StandardConcept, StandardizationCache ) # Get the singleton mapper - eliminates redundant file I/O mapper = get_default_mapper() # Map a company concept to standardized label label = mapper.map_concept( "us-gaap_AccountsPayableCurrent", "Accounts Payable", {"statement_type": "BalanceSheet"} ) # Returns: "Accounts Payable"` ### StandardConcept Enum Type-safe enum for all standardized concepts: `from edgar.xbrl.standardization import StandardConcept # Access standard concept labels StandardConcept.REVENUE.value # "Revenue" StandardConcept.NET_INCOME.value # "Net Income" StandardConcept.TOTAL_ASSETS.value # "Total Assets" StandardConcept.ACCOUNTS_PAYABLE.value # "Accounts Payable" # Look up a concept by its label concept = StandardConcept.get_from_label("Revenue") # Returns: StandardConcept.REVENUE # Get all available standard values all_values = StandardConcept.get_all_values() # Returns: {'Revenue', 'Net Income', 'Total Assets', ...}` ### StandardizationCache (Per-XBRL Caching) For high-performance workflows, use `StandardizationCache` which caches results per XBRL instance: `# The cache is automatically attached to XBRL instances xbrl = filing.xbrl() # Access via the standardization property cache = xbrl.standardization # Get cached label lookups label = cache.get_standard_label( "us-gaap_Revenue", "Total Revenue", {"statement_type": "IncomeStatement"} ) # Standardize entire statement with caching raw_data = xbrl.get_statement_data("IncomeStatement") standardized = cache.standardize_statement_data(raw_data, "IncomeStatement") # Check cache statistics print(cache.cache_stats) # {'label_cache_size': 42, 'statement_cache_size': 1, 'cached_statements': ['IncomeStatement']} # Clear cache when needed cache.clear_cache() # Clear all cache.clear_cache("IncomeStatement") # Clear specific statement` ### Reverse Index API (Low-Level) Direct O(1) lookup for XBRL tags: `from edgar.xbrl.standardization.reverse_index import ( get_standard_concept, get_display_name, lookup ) # Simple lookup concept = get_standard_concept("AccountsPayableCurrent") # Returns: "TradePayables" display = get_display_name("AccountsPayableCurrent") # Returns: "Accounts Payable" # Full result with metadata result = lookup("AccountsPayableCurrentAndNoncurrent") result.is_ambiguous # True result.standard_concepts # ['TradePayables', 'OtherOperatingNonCurrentLiabilities'] result.display_names # ['Accounts Payable', 'Other Non-Current Liabilities'] result.comment # 'Curr/NonCurr ambiguity'` ### Checking Coverage `from edgar.xbrl.standardization.reverse_index import get_reverse_index index = get_reverse_index() print(index.stats) # {'total_mappings': 2067, 'ambiguous_count': 215, 'deprecated_count': 410, 'excluded_count': 276}` * * * Context-Aware Disambiguation ---------------------------- Starting in v5.9.0, EdgarTools uses context-aware disambiguation to resolve ambiguous tags. ### How It Works Ambiguous tags (e.g., `AccountsPayableCurrentAndNoncurrent`) can map to multiple concepts. EdgarTools uses two complementary strategies to disambiguate: #### 1\. Calculation Parent Derivation When an item has a `calculation_parent`, EdgarTools infers its section: `# Item with calculation_parent="us-gaap:AssetsCurrent" # → Inferred section: "Current Assets" # → Resolves ambiguity toward current asset concepts # Supported parent → section mappings: # AssetsCurrent → "Current Assets" # AssetsNoncurrent → "Non-Current Assets" # LiabilitiesCurrent → "Current Liabilities" # LiabilitiesNoncurrent → "Non-Current Liabilities" # StockholdersEquity → "Equity"` #### 2\. Bottom-Up Section Scanning (mpreiss9 method) For items without calculation parents, EdgarTools scans the statement from bottom to top, using subtotals as section boundaries: `Total Current Assets ← Defines boundary for "Current Assets" section ↑ All items above until next subtotal belong to "Current Assets" ↑ Property, Plant & Equipment ← This item gets assigned "Current Assets" section based on its position relative to subtotal` ### Passing Context to the Mapper When calling the mapper directly, you can provide context for disambiguation: `mapper = get_default_mapper() # Provide context for accurate disambiguation label = mapper.map_concept( "us-gaap_OtherAssetsNoncurrent", "Other Assets", { "statement_type": "BalanceSheet", "section": "Non-Current Assets", # Helps resolve operating vs non-operating "calculation_parent": "us-gaap:AssetsNoncurrent", "level": 2, "is_total": False } )` ### Context Keys | Key | Description | Used For | | --- | --- | --- | | `statement_type` | "BalanceSheet", "IncomeStatement", etc. | Statement-specific matching | | `section` | "Current Assets", "Equity", etc. | Disambiguating current/non-current | | `calculation_parent` | Parent concept in calculation tree | Deriving section automatically | | `level` | Indentation level (0-5) | Identifying subtotals vs details | | `is_total` | True for subtotal/total rows | Section boundary detection | | `balance` | "debit" or "credit" | Sign-based disambiguation | * * * Files ----- | File | Purpose | | --- | --- | | `gaap_mappings.json` | 2,067 XBRL tag → standard concept mappings | | `display_names.json` | 95 standard concept → display name mappings | | `exclusions.py` | 276 excluded (DropThisItem) tags | | `reverse_index.py` | O(1) lookup implementation | | `core.py` | StandardConcept enum, MappingStore, ConceptMapper, standardize\_statement | | `cache.py` | StandardizationCache for per-XBRL instance caching | | `sections.py` | Section classification for disambiguation | | `__init__.py` | Module exports and singleton accessors | * * * Version History --------------- | Version | Date | Changes | | --- | --- | --- | | 5.9.0 | 2026-01 | Context-aware disambiguation, bottom-up section scanning, StandardizationCache | | 5.8.0 | 2026-01 | Module-level singletons (get\_default\_mapper, get\_default\_store) | | 1.0 | 2026-01 | Initial release with 95 concepts, 2,067 mappings | * * * Related Resources ----------------- * [XBRL Documentation Hub](https://edgartools.readthedocs.io/en/latest/xbrl/) - Central navigation for all XBRL docs * [Extract Financial Statements Guide](https://edgartools.readthedocs.io/en/latest/guides/extract-statements/) - Complete guide to extracting financial data * [Dimension Handling Guide](https://edgartools.readthedocs.io/en/latest/xbrl/concepts/dimension-handling/) - Understanding dimensional data and segment breakdowns * [Multi-Period Analysis Guide](https://edgartools.readthedocs.io/en/latest/xbrl/guides/multi-period-analysis/) - Working with XBRLS for multi-period comparison * * * Need help scaling XBRL standardization? EdgarTools maps 2,067 XBRL tags to 95 standard concepts. But production pipelines hit custom extensions, deprecated tags, and taxonomy version changes that go beyond standard mappings. * **[XBRL consulting for AI & data teams →](https://www.edgar.tools/consulting/xbrl?utm_source=edgartools-docs&utm_medium=see-live&utm_content=xbrl-standardization) ** * **[See all SEC data consulting services →](https://www.edgar.tools/consulting?utm_source=edgartools-docs&utm_medium=see-live&utm_content=xbrl-standardization) ** From the creator of edgartools. [Book a call →](https://www.edgar.tools/consulting/xbrl?utm_source=edgartools-docs&utm_medium=see-live&utm_content=xbrl-standardization#contact) Credits ------- The standardization taxonomy is based on the production mappings shared by [@mpreiss9](https://github.com/mpreiss9) , tested across 390+ companies. See: [GitHub Issue #494](https://github.com/dgunning/edgartools/issues/494) Back to top --- # Statement Types - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/StatementType-Quick-Reference/#statementtype-quick-reference) StatementType Quick Reference ============================= **FEAT-005: Statement Type Classifications for EdgarTools** Enhanced developer experience through IDE autocomplete and parameter validation for financial statement types. Available Statement Types ------------------------- ### Primary Financial Statements (The Big Four) | Enum Value | String Value | Description | Use Case | | --- | --- | --- | --- | | `StatementType.INCOME_STATEMENT` | `"income_statement"` | Profit & Loss Statement | Revenue, expenses, net income analysis | | `StatementType.BALANCE_SHEET` | `"balance_sheet"` | Statement of Financial Position | Assets, liabilities, equity analysis | | `StatementType.CASH_FLOW` | `"cash_flow_statement"` | Statement of Cash Flows | Cash inflows, outflows, liquidity analysis | | `StatementType.CHANGES_IN_EQUITY` | `"changes_in_equity"` | Statement of Changes in Equity | Equity movements, dividends, retained earnings | ### Comprehensive Statements | Enum Value | String Value | Description | Use Case | | --- | --- | --- | --- | | `StatementType.COMPREHENSIVE_INCOME` | `"comprehensive_income"` | Statement of Comprehensive Income | Total comprehensive income including OCI | ### Analytical Statements | Enum Value | String Value | Description | Use Case | | --- | --- | --- | --- | | `StatementType.SEGMENTS` | `"segment_reporting"` | Segment Information | Business segment performance | | `StatementType.SUBSIDIARIES` | `"subsidiaries"` | Subsidiary Information | Subsidiary company details | | `StatementType.FOOTNOTES` | `"footnotes"` | Notes to Financial Statements | Detailed disclosures and notes | | `StatementType.ACCOUNTING_POLICIES` | `"accounting_policies"` | Significant Accounting Policies | Accounting methods and principles | ### Specialized Statements | Enum Value | String Value | Description | Use Case | | --- | --- | --- | --- | | `StatementType.REGULATORY_CAPITAL` | `"regulatory_capital"` | Regulatory Capital | Bank capital adequacy ratios | | `StatementType.INSURANCE_RESERVES` | `"insurance_reserves"` | Insurance Reserves | Insurance loss reserves | ### Convenience Aliases | Alias | Same As | Notes | | --- | --- | --- | | `StatementType.PROFIT_LOSS` | `StatementType.INCOME_STATEMENT` | Common P&L terminology | | `StatementType.PL_STATEMENT` | `StatementType.INCOME_STATEMENT` | Abbreviated P&L | | `StatementType.FINANCIAL_POSITION` | `StatementType.BALANCE_SHEET` | IFRS terminology | | `StatementType.STATEMENT_OF_POSITION` | `StatementType.BALANCE_SHEET` | Alternative naming | | `StatementType.CASH_FLOWS` | `StatementType.CASH_FLOW` | Plural form | | `StatementType.EQUITY_CHANGES` | `StatementType.CHANGES_IN_EQUITY` | Shorter form | Basic Usage ----------- ### Import `from edgar.enums import StatementType, StatementInput, validate_statement_type` Two Ways to Access Financial Statements --------------------------------------- EdgarTools provides **two different APIs** for accessing financial statements, each with different use cases: ### 1\. Company Facts API (Multi-Period Historical Data) Use the **Company** class for historical financial data across multiple periods. This uses the SEC's Company Facts API. `from edgar import Company company = Company("AAPL") # Direct convenience methods (recommended for beginners) income = company.income_statement(periods=4, annual=True) balance = company.balance_sheet(periods=4, annual=True) cash = company.cashflow_statement(periods=4, annual=True) # These return MultiPeriodStatement objects with rich display print(income) # Beautiful table output` **Best for:** - Multi-period trend analysis - Quick access to historical financials - Beginners who want simple API **Limitations:** - Only supports primary statements (income, balance sheet, cash flow) - Does not support segment or analytical statements ### 2\. XBRL API (Full Statement Access) Use the **XBRL** class for complete access to all statement types from a specific filing. `from edgar import Company company = Company("AAPL") filing = company.get_filings(form="10-K").latest() xbrl = filing.xbrl() # Recommended: Use the statements property for common statements income = xbrl.statements.income_statement() balance = xbrl.statements.balance_sheet() cash_flow = xbrl.statements.cashflow_statement() # For analytical statements, use get_statement() with PascalCase string names segments = xbrl.get_statement("SegmentDisclosure") comprehensive = xbrl.get_statement("ComprehensiveIncome")` > **Note:** `get_statement()` accepts PascalCase type names (e.g., `"IncomeStatement"`, `"BalanceSheet"`, `"CashFlowStatement"`), role URIs, or statement short names — **not** `StatementType` enum values. **Best for:** - Accessing specific filing periods - Analytical statements (segments, footnotes, etc.) - Full XBRL dimensional data - Advanced analysis Accessing Segment Statements ---------------------------- Segment data is only available through the XBRL API: `from edgar import Company company = Company("AAPL") filing = company.get_filings(form="10-K").latest() xbrl = filing.xbrl() # Get segment statement data using PascalCase string name segment_data = xbrl.get_statement("SegmentDisclosure") # Segment dimensional data also appears in income statements income = xbrl.statements.income_statement() # Shows segment breakdowns by product, geography, etc. print(income)` Enhanced Validation ------------------- ### Smart Error Messages `from edgar.enums import validate_statement_type # Typo detection try: validate_statement_type("income") # partial match except ValidationError as e: # Error: "Invalid statement type 'income'. Did you mean: income_statement?" try: validate_statement_type("balanc") # misspelling except ValidationError as e: # Error: "Invalid statement type 'balanc'. Did you mean: balance_sheet?" # Context-aware help try: validate_statement_type("unknown") except ValidationError as e: # Error: "Invalid statement type 'unknown'. Primary statements: # 'income_statement' (P&L), 'balance_sheet' (financial position), ..."` Function Integration -------------------- ### Type Hints `from edgar.enums import StatementInput def analyze_statement(filing, statement: StatementInput) -> dict: """Function with StatementType parameter.""" xbrl = filing.xbrl() validated_statement = validate_statement_type(statement) # Note: get_statement() expects PascalCase names like "IncomeStatement" statement_data = xbrl.get_statement(validated_statement) return {"statement": validated_statement, "data": statement_data} # Usage with get_statement() - pass PascalCase strings result = analyze_statement(filing, "IncomeStatement") result = analyze_statement(filing, "BalanceSheet")` Convenience Collections ----------------------- `from edgar.enums import ( PRIMARY_STATEMENTS, COMPREHENSIVE_STATEMENTS, ANALYTICAL_STATEMENTS, SPECIALIZED_STATEMENTS, ALL_STATEMENTS ) # Use the statements property for primary financial statements income = xbrl.statements.income_statement() balance = xbrl.statements.balance_sheet() cash_flow = xbrl.statements.cashflow_statement() # For analytical statements, use get_statement() with PascalCase names segments = xbrl.get_statement("SegmentDisclosure") comprehensive = xbrl.get_statement("ComprehensiveIncome")` Real-World Examples ------------------- ### Financial Analysis Workflow `from edgar import Company def comprehensive_financial_analysis(ticker: str) -> dict: """Analyze company across all primary statements from latest 10-K.""" company = Company(ticker) filing = company.get_filings(form="10-K").latest() xbrl = filing.xbrl() return { "income": xbrl.statements.income_statement(), "balance": xbrl.statements.balance_sheet(), "cash_flow": xbrl.statements.cashflow_statement(), "equity": xbrl.statements.statement_of_equity(), } # Usage analysis = comprehensive_financial_analysis("AAPL")` ### Multi-Period Historical Analysis `from edgar import Company def trend_analysis(ticker: str, periods: int = 5) -> dict: """Analyze company trends using Company Facts API.""" company = Company(ticker) return { "income": company.income_statement(periods=periods, annual=True), "balance": company.balance_sheet(periods=periods, annual=True), "cash_flow": company.cashflow_statement(periods=periods, annual=True) } # Usage - returns MultiPeriodStatement objects trends = trend_analysis("AAPL", periods=5) print(trends["income"]) # Shows 5 years of income statement data` ### Statement Categorization `def categorize_available_statements(xbrl) -> dict: """Categorize available statements by type.""" categories = xbrl.statements.get_statements_by_category() return categories` IDE Benefits ------------ With StatementType, your IDE will provide: ### Autocomplete When you type `StatementType.`, your IDE shows: `StatementType.INCOME_STATEMENT # 'income_statement' - P&L Statement StatementType.BALANCE_SHEET # 'balance_sheet' - Financial Position StatementType.CASH_FLOW # 'cash_flow_statement' - Cash Flows StatementType.CHANGES_IN_EQUITY # 'changes_in_equity' - Equity Changes StatementType.COMPREHENSIVE_INCOME # 'comprehensive_income' - Total Income ...` ### Documentation Hover over enum values to see descriptions: - **INCOME\_STATEMENT**: Profit & Loss Statement showing revenues and expenses - **BALANCE\_SHEET**: Statement of Financial Position showing assets and liabilities - **CASH\_FLOW**: Statement of Cash Flows showing cash movements ### Type Safety Your IDE will warn about: - Invalid statement types - Wrong parameter types - Potential typos before runtime API Comparison -------------- | Feature | Company API | XBRL API | | --- | --- | --- | | **Methods** | `income_statement()`, `balance_sheet()`, `cash_flow()` | `xbrl.statements.income_statement()` or `xbrl.get_statement("IncomeStatement")` | | **Source** | Company Facts API | Filing XBRL data | | **Multi-Period** | Yes (built-in) | No (single filing) | | **Segments** | No | Yes | | **Footnotes** | No | Yes | | **StatementType Enum** | Not used | Not used (use PascalCase strings or `statements` property) | | **Best For** | Historical trends | Full statement access | Migration Guide --------------- ### Choosing the Right API **Use Company API when:** `# You need multi-period historical data company = Company("AAPL") income = company.income_statement(periods=5, annual=True) # 5 years of data` **Use XBRL API when:** `# You need specific filing data or analytical statements filing = company.get_filings(form="10-K").latest() xbrl = filing.xbrl() segments = xbrl.get_statement("SegmentDisclosure") # Only available here` Consistency with Other Types ---------------------------- StatementType follows the same design pattern as FormType and PeriodType: | Feature | FormType | PeriodType | StatementType | | --- | --- | --- | --- | | **Enum Type** | `StrEnum` | `StrEnum` | `StrEnum` | | **Validation** | `validate_form_type()` | `validate_period_type()` | `validate_statement_type()` | | **Type Hints** | `FormInput` | `PeriodInput` | `StatementInput` | | **Collections** | `PRIMARY_FORMS`, etc. | `STANDARD_PERIODS`, etc. | `PRIMARY_STATEMENTS`, etc. | | **Error Handling** | Smart suggestions | Smart suggestions | Smart suggestions | | **Backwards Compat** | Union types | Union types | Union types | Best Practices -------------- ### 1\. Use Appropriate API for Your Use Case `# Historical analysis - use Company API company = Company("AAPL") income = company.income_statement(periods=4) # Specific filing analysis - use XBRL statements property (recommended) xbrl = company.get_filings(form="10-K").latest().xbrl() income = xbrl.statements.income_statement() balance = xbrl.statements.balance_sheet() cash_flow = xbrl.statements.cashflow_statement()` ### 2\. Access Analytical Statements `# For statements beyond the Big Three, use get_statement() with PascalCase names xbrl = filing.xbrl() segments = xbrl.get_statement("SegmentDisclosure") comprehensive = xbrl.get_statement("ComprehensiveIncome") equity = xbrl.get_statement("StatementOfEquity")` ### 3\. Enumerate Available Statements `# See what statements are available in a filing, organized by category xbrl = filing.xbrl() categories = xbrl.statements.get_statements_by_category() for category, stmts in categories.items(): for stmt in stmts: print(f"{category}: {stmt['type']} - {stmt.get('title', '')}")` Error Handling -------------- ### Common Errors and Solutions `from edgar.enums import validate_statement_type, StatementType # Typo in string try: validate_statement_type("income") except ValidationError as e: print(e) # "Did you mean: income_statement?" # Wrong type try: validate_statement_type(123) except TypeError as e: print(e) # "Statement must be StatementType or str" # Completely invalid try: validate_statement_type("invalid_statement") except ValidationError as e: print(e) # "Use StatementType enum for autocomplete..."` * * * Impact Summary -------------- **FEAT-005 delivers on EdgarTools principles:** * **Beginner-friendly**: Makes financial statement exploration discoverable * **Simple yet powerful**: Two APIs for different use cases * **Joyful UX**: IDE autocomplete and helpful error messages **Key improvements:** - IDE autocomplete for financial statement types - Enhanced validation with financial context - Clear separation between Company and XBRL APIs - Educational categorization of statement types - Full backwards compatibility maintained - Consistent design with FormType and PeriodType Back to top --- # Period Types - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/PeriodType-Quick-Reference/#periodtype-quick-reference) PeriodType Quick Reference ========================== **FEAT-003: PeriodType Enum for EdgarTools** Enhanced developer experience through IDE autocomplete and parameter validation for financial reporting periods. 📋 Available Period Types ------------------------- | Enum Value | String Value | Description | Use Case | | --- | --- | --- | --- | | `PeriodType.ANNUAL` | `"annual"` | Annual reporting periods | Full fiscal year financial data | | `PeriodType.QUARTERLY` | `"quarterly"` | Quarterly reporting periods | 3-month period financial data | | `PeriodType.MONTHLY` | `"monthly"` | Monthly reporting periods | Monthly financial data (rare) | | `PeriodType.TTM` | `"ttm"` | Trailing Twelve Months | Rolling 12-month performance | | `PeriodType.YTD` | `"ytd"` | Year to Date | Current year performance | ### Convenience Aliases | Alias | Same As | Notes | | --- | --- | --- | | `PeriodType.YEARLY` | `PeriodType.ANNUAL` | Alternative naming | | `PeriodType.QUARTER` | `PeriodType.QUARTERLY` | Shorter form | 🚀 Basic Usage -------------- ### Import `from edgar.enums import PeriodType, PeriodInput, validate_period_type` ### Function Parameters (New Style) `from edgar import Company from edgar.enums import PeriodType # Enhanced with autocomplete for financial statements company = Company("AAPL") # NEW: Direct period type filtering in get_facts() annual_facts = company.get_facts(period_type=PeriodType.ANNUAL) quarterly_facts = company.get_facts(period_type=PeriodType.QUARTERLY) # NEW: Export facts to DataFrame for custom analysis df = annual_facts.to_dataframe() print(df.head()) # Filter and analyze revenue_facts = df[df['concept'].str.contains('Revenue')] print(revenue_facts[['fiscal_year', 'numeric_value']]) # NEW: Query interface with PeriodType enum facts = company.get_facts() annual_facts = facts.query().by_period_type(PeriodType.ANNUAL).execute() quarterly_facts = facts.query().by_period_type(PeriodType.QUARTERLY).execute() # Existing methods still work (backward compatibility) annual_income = facts.income_statement(annual=True) # Annual periods quarterly_income = facts.income_statement(annual=False) # Quarterly periods` ### Backwards Compatibility (Existing Style) `# Still works - no breaking changes from edgar import Company company = Company("AAPL") facts = company.get_facts() annual_income = facts.income_statement(annual=True) quarterly_income = facts.income_statement(annual=False)` 🛡️ Enhanced Validation ----------------------- ### Smart Error Messages `from edgar.enums import validate_period_type # Typo detection try: validate_period_type("anual") # misspelled except ValueError as e: print(e) # Error: "Invalid period type 'anual'. Did you mean: annual?" # Invalid input try: validate_period_type("invalid") except ValueError as e: print(e) # Error: "Invalid period type 'invalid'. Use PeriodType enum for autocomplete..."` 🔧 Function Integration ----------------------- ### Type Hints `from edgar.enums import PeriodInput, PeriodType, validate_period_type def analyze_financials(ticker: str, period: PeriodInput = PeriodType.ANNUAL) -> str: """Function with PeriodType parameter.""" validated_period = validate_period_type(period) return f"Analyzing {ticker} {validated_period} financials" # Usage result1 = analyze_financials("AAPL", PeriodType.QUARTERLY) # IDE autocomplete result2 = analyze_financials("MSFT", "ttm") # String still works` ### Migration from Boolean Annual `from edgar.enums import PeriodInput, PeriodType, validate_period_type # Old pattern def old_style(annual: bool = True) -> str: period = "annual" if annual else "quarterly" return f"Getting {period} data" # New pattern - more expressive def new_style(period: PeriodInput = PeriodType.ANNUAL) -> str: period_str = validate_period_type(period) return f"Getting {period_str} data" # Benefits: # ✅ Support for TTM, YTD, monthly (not just annual/quarterly) # ✅ IDE autocomplete # ✅ Validation prevents typos # ✅ Self-documenting code` 📚 Convenience Collections -------------------------- `from edgar.enums import STANDARD_PERIODS, SPECIAL_PERIODS, ALL_PERIODS # Most common periods for period in STANDARD_PERIODS: print(f"Standard: {period}") # ANNUAL, QUARTERLY # Special analysis periods for period in SPECIAL_PERIODS: print(f"Special: {period}") # TTM, YTD # All available periods for period in ALL_PERIODS: print(f"Available: {period}") # All 5 period types` 🌍 Real-World Examples ---------------------- ### Financial Analysis `from edgar.enums import PeriodInput, PeriodType def compare_performance(ticker: str, periods: list[PeriodInput]) -> dict: """Compare company performance across different periods.""" from edgar import Company company = Company(ticker) results = {} for period in periods: # NEW: Direct period filtering in get_facts() try: period_facts = company.get_facts(period_type=period) if period_facts: # Get income statement data from filtered facts income_stmt = period_facts.income_statement(periods=1) results[str(period)] = income_stmt except NotImplementedError: # Handle TTM/YTD (not yet implemented) facts = company.get_facts() if str(period) == "ttm": # Get last 4 quarters for TTM calculation data = facts.income_statement(annual=False, periods=4) results["ttm"] = data return results # Usage with mixed types analysis = compare_performance("AAPL", [ PeriodType.ANNUAL, # Enum "quarterly", # String PeriodType.MONTHLY # Enum - now supported! ]) # NEW: Enhanced query combinations company = Company("AAPL") facts = company.get_facts() # Combine period filtering with concept filtering annual_revenue = facts.query().by_period_type(PeriodType.ANNUAL).by_concept("Revenue").execute() quarterly_revenue = facts.query().by_period_type("quarterly").by_concept("Revenue").execute() # Filter specific period types directly annual_facts = facts.filter_by_period_type(PeriodType.ANNUAL) quarterly_facts = facts.filter_by_period_type(PeriodType.QUARTERLY)` ### Batch Processing `from edgar.enums import PeriodInput, PeriodType def process_companies(tickers: list[str], period: PeriodInput = PeriodType.QUARTERLY) -> dict: """Process multiple companies for specified period.""" from edgar import Company from edgar.enums import validate_period_type period_str = validate_period_type(period) results = {} for ticker in tickers: company = Company(ticker) facts = company.get_facts() statement = None # Initialize statement variable if period_str == "annual": statement = facts.income_statement(annual=True, periods=1) elif period_str == "quarterly": statement = facts.income_statement(annual=False, periods=1) if statement is not None: results[ticker] = statement return results # Usage from edgar.enums import PeriodType tech_stocks = ["AAPL", "MSFT", "GOOGL"] result = process_companies(tech_stocks, PeriodType.QUARTERLY)` ### Period Iteration `def comprehensive_analysis(ticker: str) -> dict: """Analyze company across all standard periods.""" from edgar import Company from edgar.enums import STANDARD_PERIODS company = Company(ticker) facts = company.get_facts() results = {} for period in STANDARD_PERIODS: # Each period provides IDE autocomplete when used statement = None # Initialize statement variable if period.value == "annual": statement = facts.income_statement(annual=True, periods=2) elif period.value == "quarterly": statement = facts.income_statement(annual=False, periods=4) if statement is not None: results[period.value] = statement return results` 💡 IDE Benefits --------------- With PeriodType, your IDE will provide: ### Autocomplete When you type `PeriodType.`, your IDE shows: `PeriodType.ANNUAL # 'annual' - Full fiscal year PeriodType.QUARTERLY # 'quarterly' - 3-month periods PeriodType.MONTHLY # 'monthly' - Monthly periods PeriodType.TTM # 'ttm' - Trailing twelve months PeriodType.YTD # 'ytd' - Year to date` ### Documentation Hover over enum values to see descriptions: - **ANNUAL**: Annual reporting periods (full fiscal year) - **QUARTERLY**: Quarterly reporting periods (3-month periods) - **TTM**: Trailing Twelve Months for rolling performance analysis ### Type Safety Your IDE will warn about: - Invalid period types - Wrong parameter types - Potential typos before runtime 🔄 Migration Guide ------------------ ### From Boolean Annual Parameter **Before:** `# Limited to annual/quarterly only from edgar import Company company = Company("AAPL") facts = company.get_facts() annual_income = facts.income_statement(annual=True) # Annual data quarterly_income = facts.income_statement(annual=False) # Quarterly data` **After:** `# Rich period support with enhanced querying from edgar import Company company = Company("AAPL") facts = company.get_facts() # Financial statement methods with boolean parameters annual_income = facts.income_statement(annual=True) # Annual quarterly_income = facts.income_statement(annual=False) # Quarterly # Advanced period filtering with query interface ttm_facts = facts.query().by_period_length(12).get() # Trailing twelve months quarterly_facts = facts.query().by_period_length(3).get() # Quarterly periods # Individual fact retrieval with period specification # Use XBRL concept names or lowercase labels (not simplified names like "Revenue") revenue_2024 = facts.get_fact("us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax", period="2024-FY") net_income = facts.get_fact("net income (loss) attributable to parent", period="2024-Q1")` ### From String Parameters **Before:** `# Typo-prone, no autocomplete def analyze_data(period: str) -> str: return f"Analyzing {period} data" analyze_data("annual") # Could typo as "anual" analyze_data("quarterly") # Could typo as "quartly"` **After:** `# Autocomplete prevents typos from edgar.enums import PeriodType, PeriodInput, validate_period_type def analyze_data(period: PeriodInput) -> str: validated_period = validate_period_type(period) return f"Analyzing {validated_period} data" analyze_data(PeriodType.ANNUAL) # IDE autocomplete analyze_data(PeriodType.QUARTERLY) # IDE autocomplete # Strings still work with validation analyze_data("annual") # Validated, helpful errors if typo` ⚖️ Consistency with FormType ---------------------------- PeriodType follows the same design pattern as FormType: | Feature | FormType | PeriodType | | --- | --- | --- | | **Enum Type** | `StrEnum` | `StrEnum` | | **Validation** | `validate_form_type()` | `validate_period_type()` | | **Type Hints** | `FormInput` | `PeriodInput` | | **Collections** | `PERIODIC_FORMS`, etc. | `STANDARD_PERIODS`, etc. | | **Error Handling** | Smart suggestions | Smart suggestions | | **Backwards Compat** | ✅ Union types | ✅ Union types | 🎯 Best Practices ----------------- ### 1\. Use Enums for New Code `# Recommended: Enhanced developer experience from edgar.enums import PeriodInput, PeriodType def analyze_trends(period: PeriodInput = PeriodType.ANNUAL) -> str: return f"Analyzing trends for {period}"` ### 2\. Maintain String Compatibility `# Support both for flexibility from edgar.enums import PeriodInput, validate_period_type def flexible_function(period: PeriodInput) -> str: validated = validate_period_type(period) # Handles both return f"Processing {validated} data"` ### 3\. Leverage Collections `# Use predefined collections from edgar.enums import STANDARD_PERIODS def process_period(period_type): return f"Processing {period_type}" for period in STANDARD_PERIODS: result = process_period(period) print(result)` ### 4\. Provide Good Defaults `# Use meaningful defaults from edgar.enums import PeriodInput, PeriodType def get_financials(period: PeriodInput = PeriodType.ANNUAL) -> str: """Default to annual for most financial analysis.""" return f"Getting {period} financials"` 🚦 Error Handling ----------------- ### Common Errors and Solutions `from edgar.enums import validate_period_type, PeriodType # Typo in string try: validate_period_type("anual") except ValueError as e: print(e) # "Did you mean: annual?" # Wrong type try: validate_period_type("123") # Use string instead of int except ValueError as e: print(e) # "Invalid period type '123'..." # Completely invalid try: validate_period_type("invalid") except ValueError as e: print(e) # "Use PeriodType enum for autocomplete..."` * * * 🆕 API Enhancements ------------------- **Period-Type Filtering Feature** adds direct filtering capabilities: ### New Methods `# Direct filtering in get_facts() annual_facts = company.get_facts(period_type=PeriodType.ANNUAL) # Query interface filtering facts.query().by_period_type(PeriodType.QUARTERLY) # EntityFacts filtering facts.filter_by_period_type(PeriodType.ANNUAL)` ### Enhanced Workflow `from edgar import Company from edgar.enums import PeriodType # Before: Multi-step process company = Company("AAPL") facts = company.get_facts() annual_income = facts.income_statement(annual=True) # After: Direct, intuitive filtering annual_facts = company.get_facts(period_type=PeriodType.ANNUAL) annual_income = annual_facts.income_statement() # Advanced: Query combinations revenue_annual = company.get_facts().query()\ .by_concept("Revenue")\ .by_period_type(PeriodType.ANNUAL)\ .execute()` 📈 Impact Summary ----------------- **Period-Type Filtering delivers on EdgarTools principles:** * ✅ **Simple yet powerful**: Direct filtering eliminates multi-step processes * ✅ **Beginner-friendly**: IDE autocomplete reveals filtering options * ✅ **Joyful UX**: Intuitive API that works as expected * ✅ **Accurate financials**: Type-safe period specification **Key improvements:** - 🎯 IDE autocomplete for period types - 🛡️ Enhanced validation with smart error messages - ⚡ Direct period filtering in get\_facts() - 🔍 Query interface period filtering - 🔧 Seamless integration with existing API - 🔄 Clear migration path from boolean parameters - ⚖️ Consistent design with FormType enum Back to top --- # Period Types - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/PeriodType-Quick-Reference/#periodtype-quick-reference) PeriodType Quick Reference ========================== **FEAT-003: PeriodType Enum for EdgarTools** Enhanced developer experience through IDE autocomplete and parameter validation for financial reporting periods. 📋 Available Period Types ------------------------- | Enum Value | String Value | Description | Use Case | | --- | --- | --- | --- | | `PeriodType.ANNUAL` | `"annual"` | Annual reporting periods | Full fiscal year financial data | | `PeriodType.QUARTERLY` | `"quarterly"` | Quarterly reporting periods | 3-month period financial data | | `PeriodType.MONTHLY` | `"monthly"` | Monthly reporting periods | Monthly financial data (rare) | | `PeriodType.TTM` | `"ttm"` | Trailing Twelve Months | Rolling 12-month performance | | `PeriodType.YTD` | `"ytd"` | Year to Date | Current year performance | ### Convenience Aliases | Alias | Same As | Notes | | --- | --- | --- | | `PeriodType.YEARLY` | `PeriodType.ANNUAL` | Alternative naming | | `PeriodType.QUARTER` | `PeriodType.QUARTERLY` | Shorter form | 🚀 Basic Usage -------------- ### Import `from edgar.enums import PeriodType, PeriodInput, validate_period_type` ### Function Parameters (New Style) `from edgar import Company from edgar.enums import PeriodType # Enhanced with autocomplete for financial statements company = Company("AAPL") # NEW: Direct period type filtering in get_facts() annual_facts = company.get_facts(period_type=PeriodType.ANNUAL) quarterly_facts = company.get_facts(period_type=PeriodType.QUARTERLY) # NEW: Export facts to DataFrame for custom analysis df = annual_facts.to_dataframe() print(df.head()) # Filter and analyze revenue_facts = df[df['concept'].str.contains('Revenue')] print(revenue_facts[['fiscal_year', 'numeric_value']]) # NEW: Query interface with PeriodType enum facts = company.get_facts() annual_facts = facts.query().by_period_type(PeriodType.ANNUAL).execute() quarterly_facts = facts.query().by_period_type(PeriodType.QUARTERLY).execute() # Existing methods still work (backward compatibility) annual_income = facts.income_statement(annual=True) # Annual periods quarterly_income = facts.income_statement(annual=False) # Quarterly periods` ### Backwards Compatibility (Existing Style) `# Still works - no breaking changes from edgar import Company company = Company("AAPL") facts = company.get_facts() annual_income = facts.income_statement(annual=True) quarterly_income = facts.income_statement(annual=False)` 🛡️ Enhanced Validation ----------------------- ### Smart Error Messages `from edgar.enums import validate_period_type # Typo detection try: validate_period_type("anual") # misspelled except ValueError as e: print(e) # Error: "Invalid period type 'anual'. Did you mean: annual?" # Invalid input try: validate_period_type("invalid") except ValueError as e: print(e) # Error: "Invalid period type 'invalid'. Use PeriodType enum for autocomplete..."` 🔧 Function Integration ----------------------- ### Type Hints `from edgar.enums import PeriodInput, PeriodType, validate_period_type def analyze_financials(ticker: str, period: PeriodInput = PeriodType.ANNUAL) -> str: """Function with PeriodType parameter.""" validated_period = validate_period_type(period) return f"Analyzing {ticker} {validated_period} financials" # Usage result1 = analyze_financials("AAPL", PeriodType.QUARTERLY) # IDE autocomplete result2 = analyze_financials("MSFT", "ttm") # String still works` ### Migration from Boolean Annual `from edgar.enums import PeriodInput, PeriodType, validate_period_type # Old pattern def old_style(annual: bool = True) -> str: period = "annual" if annual else "quarterly" return f"Getting {period} data" # New pattern - more expressive def new_style(period: PeriodInput = PeriodType.ANNUAL) -> str: period_str = validate_period_type(period) return f"Getting {period_str} data" # Benefits: # ✅ Support for TTM, YTD, monthly (not just annual/quarterly) # ✅ IDE autocomplete # ✅ Validation prevents typos # ✅ Self-documenting code` 📚 Convenience Collections -------------------------- `from edgar.enums import STANDARD_PERIODS, SPECIAL_PERIODS, ALL_PERIODS # Most common periods for period in STANDARD_PERIODS: print(f"Standard: {period}") # ANNUAL, QUARTERLY # Special analysis periods for period in SPECIAL_PERIODS: print(f"Special: {period}") # TTM, YTD # All available periods for period in ALL_PERIODS: print(f"Available: {period}") # All 5 period types` 🌍 Real-World Examples ---------------------- ### Financial Analysis `from edgar.enums import PeriodInput, PeriodType def compare_performance(ticker: str, periods: list[PeriodInput]) -> dict: """Compare company performance across different periods.""" from edgar import Company company = Company(ticker) results = {} for period in periods: # NEW: Direct period filtering in get_facts() try: period_facts = company.get_facts(period_type=period) if period_facts: # Get income statement data from filtered facts income_stmt = period_facts.income_statement(periods=1) results[str(period)] = income_stmt except NotImplementedError: # Handle TTM/YTD (not yet implemented) facts = company.get_facts() if str(period) == "ttm": # Get last 4 quarters for TTM calculation data = facts.income_statement(annual=False, periods=4) results["ttm"] = data return results # Usage with mixed types analysis = compare_performance("AAPL", [ PeriodType.ANNUAL, # Enum "quarterly", # String PeriodType.MONTHLY # Enum - now supported! ]) # NEW: Enhanced query combinations company = Company("AAPL") facts = company.get_facts() # Combine period filtering with concept filtering annual_revenue = facts.query().by_period_type(PeriodType.ANNUAL).by_concept("Revenue").execute() quarterly_revenue = facts.query().by_period_type("quarterly").by_concept("Revenue").execute() # Filter specific period types directly annual_facts = facts.filter_by_period_type(PeriodType.ANNUAL) quarterly_facts = facts.filter_by_period_type(PeriodType.QUARTERLY)` ### Batch Processing `from edgar.enums import PeriodInput, PeriodType def process_companies(tickers: list[str], period: PeriodInput = PeriodType.QUARTERLY) -> dict: """Process multiple companies for specified period.""" from edgar import Company from edgar.enums import validate_period_type period_str = validate_period_type(period) results = {} for ticker in tickers: company = Company(ticker) facts = company.get_facts() statement = None # Initialize statement variable if period_str == "annual": statement = facts.income_statement(annual=True, periods=1) elif period_str == "quarterly": statement = facts.income_statement(annual=False, periods=1) if statement is not None: results[ticker] = statement return results # Usage from edgar.enums import PeriodType tech_stocks = ["AAPL", "MSFT", "GOOGL"] result = process_companies(tech_stocks, PeriodType.QUARTERLY)` ### Period Iteration `def comprehensive_analysis(ticker: str) -> dict: """Analyze company across all standard periods.""" from edgar import Company from edgar.enums import STANDARD_PERIODS company = Company(ticker) facts = company.get_facts() results = {} for period in STANDARD_PERIODS: # Each period provides IDE autocomplete when used statement = None # Initialize statement variable if period.value == "annual": statement = facts.income_statement(annual=True, periods=2) elif period.value == "quarterly": statement = facts.income_statement(annual=False, periods=4) if statement is not None: results[period.value] = statement return results` 💡 IDE Benefits --------------- With PeriodType, your IDE will provide: ### Autocomplete When you type `PeriodType.`, your IDE shows: `PeriodType.ANNUAL # 'annual' - Full fiscal year PeriodType.QUARTERLY # 'quarterly' - 3-month periods PeriodType.MONTHLY # 'monthly' - Monthly periods PeriodType.TTM # 'ttm' - Trailing twelve months PeriodType.YTD # 'ytd' - Year to date` ### Documentation Hover over enum values to see descriptions: - **ANNUAL**: Annual reporting periods (full fiscal year) - **QUARTERLY**: Quarterly reporting periods (3-month periods) - **TTM**: Trailing Twelve Months for rolling performance analysis ### Type Safety Your IDE will warn about: - Invalid period types - Wrong parameter types - Potential typos before runtime 🔄 Migration Guide ------------------ ### From Boolean Annual Parameter **Before:** `# Limited to annual/quarterly only from edgar import Company company = Company("AAPL") facts = company.get_facts() annual_income = facts.income_statement(annual=True) # Annual data quarterly_income = facts.income_statement(annual=False) # Quarterly data` **After:** `# Rich period support with enhanced querying from edgar import Company company = Company("AAPL") facts = company.get_facts() # Financial statement methods with boolean parameters annual_income = facts.income_statement(annual=True) # Annual quarterly_income = facts.income_statement(annual=False) # Quarterly # Advanced period filtering with query interface ttm_facts = facts.query().by_period_length(12).get() # Trailing twelve months quarterly_facts = facts.query().by_period_length(3).get() # Quarterly periods # Individual fact retrieval with period specification # Use XBRL concept names or lowercase labels (not simplified names like "Revenue") revenue_2024 = facts.get_fact("us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax", period="2024-FY") net_income = facts.get_fact("net income (loss) attributable to parent", period="2024-Q1")` ### From String Parameters **Before:** `# Typo-prone, no autocomplete def analyze_data(period: str) -> str: return f"Analyzing {period} data" analyze_data("annual") # Could typo as "anual" analyze_data("quarterly") # Could typo as "quartly"` **After:** `# Autocomplete prevents typos from edgar.enums import PeriodType, PeriodInput, validate_period_type def analyze_data(period: PeriodInput) -> str: validated_period = validate_period_type(period) return f"Analyzing {validated_period} data" analyze_data(PeriodType.ANNUAL) # IDE autocomplete analyze_data(PeriodType.QUARTERLY) # IDE autocomplete # Strings still work with validation analyze_data("annual") # Validated, helpful errors if typo` ⚖️ Consistency with FormType ---------------------------- PeriodType follows the same design pattern as FormType: | Feature | FormType | PeriodType | | --- | --- | --- | | **Enum Type** | `StrEnum` | `StrEnum` | | **Validation** | `validate_form_type()` | `validate_period_type()` | | **Type Hints** | `FormInput` | `PeriodInput` | | **Collections** | `PERIODIC_FORMS`, etc. | `STANDARD_PERIODS`, etc. | | **Error Handling** | Smart suggestions | Smart suggestions | | **Backwards Compat** | ✅ Union types | ✅ Union types | 🎯 Best Practices ----------------- ### 1\. Use Enums for New Code `# Recommended: Enhanced developer experience from edgar.enums import PeriodInput, PeriodType def analyze_trends(period: PeriodInput = PeriodType.ANNUAL) -> str: return f"Analyzing trends for {period}"` ### 2\. Maintain String Compatibility `# Support both for flexibility from edgar.enums import PeriodInput, validate_period_type def flexible_function(period: PeriodInput) -> str: validated = validate_period_type(period) # Handles both return f"Processing {validated} data"` ### 3\. Leverage Collections `# Use predefined collections from edgar.enums import STANDARD_PERIODS def process_period(period_type): return f"Processing {period_type}" for period in STANDARD_PERIODS: result = process_period(period) print(result)` ### 4\. Provide Good Defaults `# Use meaningful defaults from edgar.enums import PeriodInput, PeriodType def get_financials(period: PeriodInput = PeriodType.ANNUAL) -> str: """Default to annual for most financial analysis.""" return f"Getting {period} financials"` 🚦 Error Handling ----------------- ### Common Errors and Solutions `from edgar.enums import validate_period_type, PeriodType # Typo in string try: validate_period_type("anual") except ValueError as e: print(e) # "Did you mean: annual?" # Wrong type try: validate_period_type("123") # Use string instead of int except ValueError as e: print(e) # "Invalid period type '123'..." # Completely invalid try: validate_period_type("invalid") except ValueError as e: print(e) # "Use PeriodType enum for autocomplete..."` * * * 🆕 API Enhancements ------------------- **Period-Type Filtering Feature** adds direct filtering capabilities: ### New Methods `# Direct filtering in get_facts() annual_facts = company.get_facts(period_type=PeriodType.ANNUAL) # Query interface filtering facts.query().by_period_type(PeriodType.QUARTERLY) # EntityFacts filtering facts.filter_by_period_type(PeriodType.ANNUAL)` ### Enhanced Workflow `from edgar import Company from edgar.enums import PeriodType # Before: Multi-step process company = Company("AAPL") facts = company.get_facts() annual_income = facts.income_statement(annual=True) # After: Direct, intuitive filtering annual_facts = company.get_facts(period_type=PeriodType.ANNUAL) annual_income = annual_facts.income_statement() # Advanced: Query combinations revenue_annual = company.get_facts().query()\ .by_concept("Revenue")\ .by_period_type(PeriodType.ANNUAL)\ .execute()` 📈 Impact Summary ----------------- **Period-Type Filtering delivers on EdgarTools principles:** * ✅ **Simple yet powerful**: Direct filtering eliminates multi-step processes * ✅ **Beginner-friendly**: IDE autocomplete reveals filtering options * ✅ **Joyful UX**: Intuitive API that works as expected * ✅ **Accurate financials**: Type-safe period specification **Key improvements:** - 🎯 IDE autocomplete for period types - 🛡️ Enhanced validation with smart error messages - ⚡ Direct period filtering in get\_facts() - 🔍 Query interface period filtering - 🔧 Seamless integration with existing API - 🔄 Clear migration path from boolean parameters - ⚖️ Consistent design with FormType enum Back to top --- # Current Filings - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/current-filings/#get-todays-sec-filings-real-time-edgar-filing-access) Get Today's SEC Filings: Real-Time EDGAR Filing Access ====================================================== Overview -------- Current filings represent the most recently submitted documents to the SEC, updated in real-time as companies file their reports. This guide shows you how to access, filter, and efficiently process current filings using edgartools. Quick Start ----------- ### Basic Usage `from edgar import get_current_filings # Get the most recent filings (default: 100 filings) current = get_current_filings() print(f"Found {len(current)} recent filings") # Display the first few filings for filing in current[:5]: print(f"{filing.form}: {filing.company} - {filing.filing_date}")` **Output:** `Found 100 recent filings 8-K: Apple Inc. - 2025-01-14 10-Q: Microsoft Corporation - 2025-01-14 4: BEZOS JEFFREY P - 2025-01-14 13F-HR: Berkshire Hathaway Inc - 2025-01-14 S-3: Tesla, Inc. - 2025-01-14` ### Filter by Form Type `# Get only Form 8-K current events current_8k = get_current_filings(form='8-K') # Get only insider trading forms (Forms 3, 4, 5) current_insider = get_current_filings(form='4') # Get quarterly and annual reports current_reports = get_current_filings(form='10-K')` Understanding Current Filings ----------------------------- ### What Are Current Filings? Current filings are the most recently submitted documents to the SEC, typically updated every few minutes during business hours. They include: * **Form 8-K**: Current events and corporate changes * **Forms 3, 4, 5**: Insider trading transactions * **10-K/10-Q**: Annual and quarterly reports * **13F**: Institutional investment manager holdings * **S-1, S-3**: Registration statements * **And many more...** ### Pagination System Current filings are delivered in pages to manage large volumes: `# Default: Get first 100 filings current = get_current_filings(page_size=100) # Get more filings per page (up to 100) current = get_current_filings(page_size=80) # Navigate to next page next_page = current.next() if next_page: print(f"Next page has {len(next_page)} filings")` Core Functions -------------- ### `get_current_filings()` Get a single page of current filings with filtering options. `def get_current_filings(form: str = '', owner: str = 'include', page_size: int = 100) -> CurrentFilings:` **Parameters:** - `form` (str): Filter by form type (e.g., "8-K", "10-K", "4") - `owner` (str): Owner filter - "include", "exclude", or "only" - `page_size` (int): Filings per page (10, 20, 40, 80, or 100) **Returns:** `CurrentFilings` object with pagination capabilities ### `iter_current_filings_pages()` Iterator that yields pages of current filings until exhausted. `from edgar import iter_current_filings_pages # Process all current 8-K filings page by page for page in iter_current_filings_pages(form="8-K"): print(f"Processing {len(page)} 8-K filings") for filing in page: # Process each filing print(f" {filing.company}: {filing.filing_date}") # Break after first few pages for demo if page.current_page >= 3: break` ### `get_all_current_filings()` Get ALL current filings by automatically iterating through all pages. `from edgar import get_all_current_filings # Get all current Form 4 filings (may be thousands) all_form4 = get_all_current_filings(form="4") print(f"Total Form 4 filings: {len(all_form4)}") # Get all current filings (no form filter) all_current = get_all_current_filings() print(f"Total current filings: {len(all_current)}")` **⚠️ Performance Note:** This function downloads ALL available current filings, which can be thousands of documents. Use with appropriate filters. Filtering Options ----------------- ### By Form Type `# Specific form types form_8k = get_current_filings(form="8-K") form_10k = get_current_filings(form="10-K") form_4 = get_current_filings(form="4") # Form families work too quarterly_reports = get_current_filings(form="10-Q")` ### By Owner Type Control whether to include filings from investment managers: `# Include all filings (default) all_filings = get_current_filings(owner="include") # Exclude ownership filings (e.g., Form 4, 144) public_only = get_current_filings(owner="exclude") # Only ownership filings (e.g., Form 4, 144) managers_only = get_current_filings(owner="only")` ### By Page Size Choose how many filings to get per request: `# Small batches for quick processing small_batch = get_current_filings(page_size=20) # Large batches for efficiency large_batch = get_current_filings(page_size=100) # Maximum` Real-World Examples ------------------- ### Example 1: Monitor Recent 8-K Events `from edgar import get_all_current_filings from datetime import datetime def monitor_current_events(): """Monitor recent 8-K filings for significant events.""" # Get recent 8-K filings current_8k = get_all_current_filings(form="8-K") print(f"📈 Monitoring {len(current_8k)} recent 8-K filings") print(f"Last updated: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}") print("-" * 60) for filing in current_8k: # Show key information print(f"{filing.company}") print(f" Form: {filing.form}") print(f" Filed: {filing.filing_date}") print(f" URL: {filing.document_url}") print() monitor_current_events()` ### Example 2: Track Insider Trading Activity `from edgar import get_all_current_filings import pandas as pd def analyze_insider_activity(): """Analyze current insider trading patterns.""" # Get all current Form 4 filings print("📊 Downloading all current Form 4 filings...") insider_filings = get_all_current_filings(form="4") print(f"Found {len(insider_filings)} insider trading filings") # Convert to DataFrame for analysis df = insider_filings.to_pandas() # Analyze by company company_counts = df['company'].value_counts().head(10) print("\n🏢 Top 10 Companies by Filing Volume:") for company, count in company_counts.items(): print(f" {company}: {count} filings") # Analyze by filing date daily_counts = df['filing_date'].value_counts().sort_index() print(f"\n📅 Daily Filing Counts (last {len(daily_counts)} days):") for date, count in daily_counts.tail(7).items(): print(f" {date}: {count} filings") return df # Run the analysis insider_df = analyze_insider_activity()` ### Example 3: Real-Time Filing Feed `from edgar import get_current_filings import time def real_time_filing_feed(max_iterations=10): """Create a real-time feed of new filings.""" seen_filings = set() iteration = 0 print("🔄 Starting real-time filing feed...") print("Press Ctrl+C to stop\n") try: while iteration < max_iterations: # Get latest filings current = get_current_filings(page_size=20) new_filings = [] for filing in current: filing_id = filing.accession_no if filing_id not in seen_filings: new_filings.append(filing) seen_filings.add(filing_id) if new_filings: print(f"🆕 {len(new_filings)} new filings detected:") for filing in new_filings: print(f" {filing.form}: {filing.company}") print() else: print("⏳ No new filings found, waiting...") # Wait before next check time.sleep(30) # Check every 30 seconds iteration += 1 except KeyboardInterrupt: print("\n✋ Feed stopped by user") # Run the feed (limited iterations for demo) real_time_filing_feed()` Performance Considerations -------------------------- ### Memory Usage `# Memory efficient: Process page by page total_processed = 0 for page in iter_current_filings_pages(form="8-K"): # Process this page total_processed += len(page) # Page goes out of scope, memory is freed print(f"Processed {total_processed} total filings") # Memory intensive: Load all at once all_filings = get_all_current_filings() # May use significant memory` ### Network Efficiency `# Efficient: Larger page sizes reduce requests efficient = get_current_filings(page_size=100) # 1 request # Less efficient: Smaller pages mean more requests less_efficient = get_current_filings(page_size=10) # May need 10 requests for same data` ### Rate Limiting The SEC imposes rate limits, so avoid rapid consecutive requests: `import time # Good: Natural pacing between requests for page in iter_current_filings_pages(): # Process page time.sleep(0.1) # Brief pause between pages # Bad: Rapid fire requests (may hit rate limits) for i in range(100): page = get_current_filings() # Don't do this!` Choosing the Right Function --------------------------- ### Use `get_current_filings()` when: * ✅ You want a quick sample of recent filings * ✅ Building pagination in your own interface * ✅ Memory usage is a concern * ✅ You only need the first page or two ### Use `iter_current_filings_pages()` when: * ✅ You want to process all filings but control memory usage * ✅ You need page-by-page processing logic * ✅ You want to limit total pages processed * ✅ Building streaming or incremental processing ### Use `get_all_current_filings()` when: * ✅ You need the complete dataset for analysis * ✅ Memory usage is not a constraint * ✅ You want to convert to pandas DataFrame * ✅ Building bulk analysis or reporting Error Handling -------------- ### Common Issues and Solutions `from edgar import get_current_filings import time def robust_current_filings(form="", max_retries=3): """Get current filings with error handling.""" for attempt in range(max_retries): try: return get_current_filings(form=form) except ConnectionError as e: print(f"⚠️ Connection error (attempt {attempt + 1}): {e}") if attempt < max_retries - 1: time.sleep(2 ** attempt) # Exponential backoff else: raise except Exception as e: print(f"❌ Unexpected error: {e}") raise # Usage try: filings = robust_current_filings(form="8-K") print(f"✅ Successfully retrieved {len(filings)} filings") except Exception as e: print(f"💥 Failed to get filings: {e}")` Best Practices -------------- ### 1\. Use Appropriate Filters `# Good: Specific filtering reduces data and improves performance insider_filings = get_current_filings(form="4") corporate_events = get_current_filings(form="8-K") # Okay: General purpose but processes more data all_filings = get_current_filings()` ### 2\. Handle Pagination Properly `# Good: Check for None before processing next page current_page = get_current_filings() while current_page is not None: # Process current page for filing in current_page: print(f"Processing {filing.company}") # Get next page current_page = current_page.next() # Bad: Assuming next() always returns data # This could cause infinite loops or errors` ### 3\. Be Respectful of SEC Resources `# Good: Process in reasonable batches with pauses for page in iter_current_filings_pages(page_size=100): # Process page time.sleep(0.1) # Brief pause # Good: Cache results when possible cached_filings = get_all_current_filings(form="8-K") # Reuse cached_filings instead of re-downloading` Common Use Cases ---------------- ### Research and Analysis * **Market surveillance**: Monitor 8-K filings for material events * **Insider tracking**: Analyze Form 4 patterns for trading insights * **Compliance monitoring**: Track filing compliance across companies ### Application Development * **Filing alerts**: Build notifications for specific form types * **Data pipelines**: Integrate current filings into larger workflows * **Dashboard feeds**: Power real-time filing displays ### Academic Research * **Event studies**: Analyze market reactions to filing events * **Disclosure analysis**: Study timing and content patterns * **Regulatory compliance**: Research filing behavior patterns Summary ------- Current filings provide real-time access to the latest SEC documents, enabling immediate analysis of corporate events, insider trading, and regulatory submissions. The three main functions offer flexibility for different use cases: * **`get_current_filings()`**: Single page access with pagination control * **`iter_current_filings_pages()`**: Memory-efficient iteration through all pages * **`get_all_current_filings()`**: Bulk access to complete current filing dataset Choose the approach that best fits your memory constraints, processing requirements, and analysis goals. Next Steps ---------- * **Guide**: [Working with Filings](https://edgartools.readthedocs.io/en/latest/guides/current-filings/working-with-filings.md) * **API Reference**: [Filings API](https://edgartools.readthedocs.io/en/latest/api/filings/) Back to top --- # Institutional Holdings (13F) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/thirteenf-data-object-guide/#13f-holdings-parse-sec-institutional-portfolio-filings-with-python) 13F Holdings: Parse SEC Institutional Portfolio Filings with Python =================================================================== See what the big funds are buying. SEC 13F filings disclose the equity holdings of institutional managers with over $100M in assets -- every quarter, publicly available. EdgarTools parses these filings into structured Python objects so you can analyze portfolios in a few lines of code. `from edgar import get_filings filings = get_filings(form="13F-HR") report = filings[0].obj() report` ![13F holdings report parsed with Python edgartools](https://edgartools.readthedocs.io/en/stable/images/thirteenf.webp) Three lines to get a fully parsed holdings report with management company, total portfolio value, and every position. * * * Access Holdings Data -------------------- The `.holdings` property returns a DataFrame with one row per security, aggregated across managers, sorted by value: `report.holdings` | Column | What it is | | --- | --- | | `Issuer` | Company name (`"APPLE INC"`) | | `Ticker` | Resolved ticker symbol (`"AAPL"`) | | `Value` | Market value in **thousands** of dollars | | `SharesPrnAmount` | Share count or principal amount | | `Cusip` | 9-character CUSIP | | `Type` | `"Shares"` or `"Principal"` | | `PutCall` | `"PUT"`, `"CALL"`, or empty | Values are in thousands -- the SEC's reporting unit. `Value` of 135,364 means $135.4 million. * * * Compare 13F Holdings Quarter-over-Quarter ----------------------------------------- One call to see what changed: `report.compare_holdings()` ![Python 13F holdings quarter-over-quarter comparison](https://edgartools.readthedocs.io/en/stable/images/thirteenf_compare.webp) Every position gets a status: **NEW**, **CLOSED**, **INCREASED**, **DECREASED**, or **UNCHANGED**. Results are sorted by absolute value change so the biggest moves appear first. `comparison = report.compare_holdings() # Dig into the data df = comparison.data new_buys = df[df['Status'] == 'NEW'] exits = df[df['Status'] == 'CLOSED']` The comparison DataFrame includes `Shares`, `PrevShares`, `ShareChange`, `ShareChangePct`, `Value`, `PrevValue`, `ValueChange`, `ValueChangePct`, and `Status`. * * * Track Holdings Trends Across Quarters ------------------------------------- See how positions evolve across quarters with sparkline visualizations: `report.holding_history(periods=4)` ![13F holdings history with sparkline trends in Python](https://edgartools.readthedocs.io/en/stable/images/thirteenf_history.webp) Each row shows share counts per quarter and a Unicode sparkline (`▁▂▃▅▇`) so you can spot trends at a glance. `history = report.holding_history(periods=4) df = history.data # Full DataFrame with one column per quarter` * * * Using View Objects in Your Own App ---------------------------------- `holdings_view()`, `compare_holdings()`, and `holding_history()` all return view objects that render in the terminal via Rich but also support iteration, indexing, and access to the underlying DataFrame. This makes them useful for building your own dashboards, reports, or exports. All three views share the same interface: `view = report.holdings_view() comparison = report.compare_holdings() history = report.holding_history(periods=4) # Iterate rows as dicts for row in view: print(row['Ticker'], row['Value']) # Index a single row (returns dict) view[0] # Slice (returns DataFrame) view[:10] # Length len(view) # Access the full DataFrame directly view.data comparison.data history.data` Each view also carries metadata useful for rendering headers: | View | Metadata | | --- | --- | | `HoldingsView` | `.display_limit` | | `HoldingsComparison` | `.current_period`, `.previous_period`, `.manager_name` | | `HoldingsHistory` | `.periods` (list of quarter dates), `.manager_name` | * * * Look Up a Specific Fund ----------------------- `from edgar import Company berkshire = Company("BRK.A") filing = berkshire.get_filings(form="13F-HR").latest(1) report = filing.obj() print(report.management_company_name) # "Berkshire Hathaway Inc" print(f"${report.total_value:,}K across {report.total_holdings} holdings")` * * * Common Analysis Patterns ------------------------ ### Portfolio concentration `h = report.holdings total = h['Value'].sum() h['Weight'] = (h['Value'] / total * 100).round(2) h[['Ticker', 'Issuer', 'Value', 'Weight']].head(10)` ### Options positions `report.holdings.query("PutCall in ['PUT', 'CALL']")` ### Previous quarter's full report `previous = report.previous_holding_report() # Returns a ThirteenF or None previous.holdings` * * * Multi-Manager Filings --------------------- Large institutions (Bank of America, State Street) file consolidated 13F reports. The `holdings` property automatically aggregates across all managers. If you need per-manager detail, use `infotable` instead: `report.infotable # Disaggregated: one row per manager-security pair report.holdings # Aggregated: one row per security (recommended) # Example: Berkshire Hathaway # infotable: ~121 rows (3 managers x ~40 securities) # holdings: ~40 rows (aggregated by CUSIP) # See who the other managers are for mgr in report.other_managers: print(f"{mgr.name} (CIK: {mgr.cik})")` * * * Metadata Quick Reference ------------------------ | Property | Returns | Example | | --- | --- | --- | | `management_company_name` | Company that filed | `"Berkshire Hathaway Inc"` | | `report_period` | Quarter end date | `"2024-03-31"` | | `filing_date` | Date filed | `"2024-05-15"` | | `total_value` | Portfolio value ($000s) | `Decimal('313218000')` | | `total_holdings` | Number of positions | `40` | | `filing_signer_name` | Who signed | `"Marc D. Hamburg"` | | `filing_signer_title` | Signer's title | `"Senior Vice President"` | | `form` | Form type | `"13F-HR"` | | `accession_number` | SEC accession no. | `"0000950123-24-007092"` | | `has_infotable()` | Has holdings data? | `True` for 13F-HR, `False` for 13F-NT | * * * Methods Quick Reference ----------------------- | Call | Returns | What it does | | --- | --- | --- | | `report.holdings` | `DataFrame` | Aggregated holdings, one row per security | | `report.infotable` | `DataFrame` | Raw holdings, disaggregated by manager | | `report.holdings_view()` | `HoldingsView` | Rich-renderable, iterable holdings | | `report.compare_holdings()` | `HoldingsComparison` | Quarter-over-quarter changes with status labels | | `report.holding_history(periods=4)` | `HoldingsHistory` | Multi-quarter share trends with sparklines | | `report.previous_holding_report()` | `ThirteenF` | Previous quarter's 13F object | | `report.other_managers` | `list[OtherManager]` | Affiliated managers in consolidated filings | | `report.get_portfolio_managers()` | `list[dict]` | Curated lookup of known portfolio managers | * * * Things to Know -------------- **Values are in thousands.** The SEC requires 13F values in $000s. A `Value` of 135,364 is $135.4 million. **`holdings` vs `infotable`.** Use `holdings` (aggregated by CUSIP) for portfolio analysis. Use `infotable` only when you need per-manager detail in multi-manager filings. **Ticker resolution.** Tickers are resolved from CUSIPs. Most resolve correctly, but delisted or obscure securities may show as blank. **Pre-2013 filings use TXT format.** EdgarTools parses both XML (2013+) and TXT (2012 and earlier) transparently, but older filings may have fewer columns. **13F-NT means no holdings.** Notice filings indicate the manager had nothing to report. `has_infotable()` returns `False`. **Report period vs filing date.** The `report_period` is the quarter end. The `filing_date` can be up to 45 days later. Some managers file multiple historical periods on the same day. * * * Related ------- * [Institutional Holdings Guide](https://edgartools.readthedocs.io/en/stable/13f-filings/) -- workflow-oriented guide for finding, analyzing, and comparing 13F holdings * [Working with Filings](https://edgartools.readthedocs.io/en/stable/guides/working-with-filing/) -- general filing access patterns Back to top --- # Statement Types - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/StatementType-Quick-Reference/#statementtype-quick-reference) StatementType Quick Reference ============================= **FEAT-005: Statement Type Classifications for EdgarTools** Enhanced developer experience through IDE autocomplete and parameter validation for financial statement types. Available Statement Types ------------------------- ### Primary Financial Statements (The Big Four) | Enum Value | String Value | Description | Use Case | | --- | --- | --- | --- | | `StatementType.INCOME_STATEMENT` | `"income_statement"` | Profit & Loss Statement | Revenue, expenses, net income analysis | | `StatementType.BALANCE_SHEET` | `"balance_sheet"` | Statement of Financial Position | Assets, liabilities, equity analysis | | `StatementType.CASH_FLOW` | `"cash_flow_statement"` | Statement of Cash Flows | Cash inflows, outflows, liquidity analysis | | `StatementType.CHANGES_IN_EQUITY` | `"changes_in_equity"` | Statement of Changes in Equity | Equity movements, dividends, retained earnings | ### Comprehensive Statements | Enum Value | String Value | Description | Use Case | | --- | --- | --- | --- | | `StatementType.COMPREHENSIVE_INCOME` | `"comprehensive_income"` | Statement of Comprehensive Income | Total comprehensive income including OCI | ### Analytical Statements | Enum Value | String Value | Description | Use Case | | --- | --- | --- | --- | | `StatementType.SEGMENTS` | `"segment_reporting"` | Segment Information | Business segment performance | | `StatementType.SUBSIDIARIES` | `"subsidiaries"` | Subsidiary Information | Subsidiary company details | | `StatementType.FOOTNOTES` | `"footnotes"` | Notes to Financial Statements | Detailed disclosures and notes | | `StatementType.ACCOUNTING_POLICIES` | `"accounting_policies"` | Significant Accounting Policies | Accounting methods and principles | ### Specialized Statements | Enum Value | String Value | Description | Use Case | | --- | --- | --- | --- | | `StatementType.REGULATORY_CAPITAL` | `"regulatory_capital"` | Regulatory Capital | Bank capital adequacy ratios | | `StatementType.INSURANCE_RESERVES` | `"insurance_reserves"` | Insurance Reserves | Insurance loss reserves | ### Convenience Aliases | Alias | Same As | Notes | | --- | --- | --- | | `StatementType.PROFIT_LOSS` | `StatementType.INCOME_STATEMENT` | Common P&L terminology | | `StatementType.PL_STATEMENT` | `StatementType.INCOME_STATEMENT` | Abbreviated P&L | | `StatementType.FINANCIAL_POSITION` | `StatementType.BALANCE_SHEET` | IFRS terminology | | `StatementType.STATEMENT_OF_POSITION` | `StatementType.BALANCE_SHEET` | Alternative naming | | `StatementType.CASH_FLOWS` | `StatementType.CASH_FLOW` | Plural form | | `StatementType.EQUITY_CHANGES` | `StatementType.CHANGES_IN_EQUITY` | Shorter form | Basic Usage ----------- ### Import `from edgar.enums import StatementType, StatementInput, validate_statement_type` Two Ways to Access Financial Statements --------------------------------------- EdgarTools provides **two different APIs** for accessing financial statements, each with different use cases: ### 1\. Company Facts API (Multi-Period Historical Data) Use the **Company** class for historical financial data across multiple periods. This uses the SEC's Company Facts API. `from edgar import Company company = Company("AAPL") # Direct convenience methods (recommended for beginners) income = company.income_statement(periods=4, annual=True) balance = company.balance_sheet(periods=4, annual=True) cash = company.cashflow_statement(periods=4, annual=True) # These return MultiPeriodStatement objects with rich display print(income) # Beautiful table output` **Best for:** - Multi-period trend analysis - Quick access to historical financials - Beginners who want simple API **Limitations:** - Only supports primary statements (income, balance sheet, cash flow) - Does not support segment or analytical statements ### 2\. XBRL API (Full Statement Access) Use the **XBRL** class for complete access to all statement types from a specific filing. `from edgar import Company company = Company("AAPL") filing = company.get_filings(form="10-K").latest() xbrl = filing.xbrl() # Recommended: Use the statements property for common statements income = xbrl.statements.income_statement() balance = xbrl.statements.balance_sheet() cash_flow = xbrl.statements.cashflow_statement() # For analytical statements, use get_statement() with PascalCase string names segments = xbrl.get_statement("SegmentDisclosure") comprehensive = xbrl.get_statement("ComprehensiveIncome")` > **Note:** `get_statement()` accepts PascalCase type names (e.g., `"IncomeStatement"`, `"BalanceSheet"`, `"CashFlowStatement"`), role URIs, or statement short names — **not** `StatementType` enum values. **Best for:** - Accessing specific filing periods - Analytical statements (segments, footnotes, etc.) - Full XBRL dimensional data - Advanced analysis Accessing Segment Statements ---------------------------- Segment data is only available through the XBRL API: `from edgar import Company company = Company("AAPL") filing = company.get_filings(form="10-K").latest() xbrl = filing.xbrl() # Get segment statement data using PascalCase string name segment_data = xbrl.get_statement("SegmentDisclosure") # Segment dimensional data also appears in income statements income = xbrl.statements.income_statement() # Shows segment breakdowns by product, geography, etc. print(income)` Enhanced Validation ------------------- ### Smart Error Messages `from edgar.enums import validate_statement_type # Typo detection try: validate_statement_type("income") # partial match except ValidationError as e: # Error: "Invalid statement type 'income'. Did you mean: income_statement?" try: validate_statement_type("balanc") # misspelling except ValidationError as e: # Error: "Invalid statement type 'balanc'. Did you mean: balance_sheet?" # Context-aware help try: validate_statement_type("unknown") except ValidationError as e: # Error: "Invalid statement type 'unknown'. Primary statements: # 'income_statement' (P&L), 'balance_sheet' (financial position), ..."` Function Integration -------------------- ### Type Hints `from edgar.enums import StatementInput def analyze_statement(filing, statement: StatementInput) -> dict: """Function with StatementType parameter.""" xbrl = filing.xbrl() validated_statement = validate_statement_type(statement) # Note: get_statement() expects PascalCase names like "IncomeStatement" statement_data = xbrl.get_statement(validated_statement) return {"statement": validated_statement, "data": statement_data} # Usage with get_statement() - pass PascalCase strings result = analyze_statement(filing, "IncomeStatement") result = analyze_statement(filing, "BalanceSheet")` Convenience Collections ----------------------- `from edgar.enums import ( PRIMARY_STATEMENTS, COMPREHENSIVE_STATEMENTS, ANALYTICAL_STATEMENTS, SPECIALIZED_STATEMENTS, ALL_STATEMENTS ) # Use the statements property for primary financial statements income = xbrl.statements.income_statement() balance = xbrl.statements.balance_sheet() cash_flow = xbrl.statements.cashflow_statement() # For analytical statements, use get_statement() with PascalCase names segments = xbrl.get_statement("SegmentDisclosure") comprehensive = xbrl.get_statement("ComprehensiveIncome")` Real-World Examples ------------------- ### Financial Analysis Workflow `from edgar import Company def comprehensive_financial_analysis(ticker: str) -> dict: """Analyze company across all primary statements from latest 10-K.""" company = Company(ticker) filing = company.get_filings(form="10-K").latest() xbrl = filing.xbrl() return { "income": xbrl.statements.income_statement(), "balance": xbrl.statements.balance_sheet(), "cash_flow": xbrl.statements.cashflow_statement(), "equity": xbrl.statements.statement_of_equity(), } # Usage analysis = comprehensive_financial_analysis("AAPL")` ### Multi-Period Historical Analysis `from edgar import Company def trend_analysis(ticker: str, periods: int = 5) -> dict: """Analyze company trends using Company Facts API.""" company = Company(ticker) return { "income": company.income_statement(periods=periods, annual=True), "balance": company.balance_sheet(periods=periods, annual=True), "cash_flow": company.cashflow_statement(periods=periods, annual=True) } # Usage - returns MultiPeriodStatement objects trends = trend_analysis("AAPL", periods=5) print(trends["income"]) # Shows 5 years of income statement data` ### Statement Categorization `def categorize_available_statements(xbrl) -> dict: """Categorize available statements by type.""" categories = xbrl.statements.get_statements_by_category() return categories` IDE Benefits ------------ With StatementType, your IDE will provide: ### Autocomplete When you type `StatementType.`, your IDE shows: `StatementType.INCOME_STATEMENT # 'income_statement' - P&L Statement StatementType.BALANCE_SHEET # 'balance_sheet' - Financial Position StatementType.CASH_FLOW # 'cash_flow_statement' - Cash Flows StatementType.CHANGES_IN_EQUITY # 'changes_in_equity' - Equity Changes StatementType.COMPREHENSIVE_INCOME # 'comprehensive_income' - Total Income ...` ### Documentation Hover over enum values to see descriptions: - **INCOME\_STATEMENT**: Profit & Loss Statement showing revenues and expenses - **BALANCE\_SHEET**: Statement of Financial Position showing assets and liabilities - **CASH\_FLOW**: Statement of Cash Flows showing cash movements ### Type Safety Your IDE will warn about: - Invalid statement types - Wrong parameter types - Potential typos before runtime API Comparison -------------- | Feature | Company API | XBRL API | | --- | --- | --- | | **Methods** | `income_statement()`, `balance_sheet()`, `cash_flow()` | `xbrl.statements.income_statement()` or `xbrl.get_statement("IncomeStatement")` | | **Source** | Company Facts API | Filing XBRL data | | **Multi-Period** | Yes (built-in) | No (single filing) | | **Segments** | No | Yes | | **Footnotes** | No | Yes | | **StatementType Enum** | Not used | Not used (use PascalCase strings or `statements` property) | | **Best For** | Historical trends | Full statement access | Migration Guide --------------- ### Choosing the Right API **Use Company API when:** `# You need multi-period historical data company = Company("AAPL") income = company.income_statement(periods=5, annual=True) # 5 years of data` **Use XBRL API when:** `# You need specific filing data or analytical statements filing = company.get_filings(form="10-K").latest() xbrl = filing.xbrl() segments = xbrl.get_statement("SegmentDisclosure") # Only available here` Consistency with Other Types ---------------------------- StatementType follows the same design pattern as FormType and PeriodType: | Feature | FormType | PeriodType | StatementType | | --- | --- | --- | --- | | **Enum Type** | `StrEnum` | `StrEnum` | `StrEnum` | | **Validation** | `validate_form_type()` | `validate_period_type()` | `validate_statement_type()` | | **Type Hints** | `FormInput` | `PeriodInput` | `StatementInput` | | **Collections** | `PRIMARY_FORMS`, etc. | `STANDARD_PERIODS`, etc. | `PRIMARY_STATEMENTS`, etc. | | **Error Handling** | Smart suggestions | Smart suggestions | Smart suggestions | | **Backwards Compat** | Union types | Union types | Union types | Best Practices -------------- ### 1\. Use Appropriate API for Your Use Case `# Historical analysis - use Company API company = Company("AAPL") income = company.income_statement(periods=4) # Specific filing analysis - use XBRL statements property (recommended) xbrl = company.get_filings(form="10-K").latest().xbrl() income = xbrl.statements.income_statement() balance = xbrl.statements.balance_sheet() cash_flow = xbrl.statements.cashflow_statement()` ### 2\. Access Analytical Statements `# For statements beyond the Big Three, use get_statement() with PascalCase names xbrl = filing.xbrl() segments = xbrl.get_statement("SegmentDisclosure") comprehensive = xbrl.get_statement("ComprehensiveIncome") equity = xbrl.get_statement("StatementOfEquity")` ### 3\. Enumerate Available Statements `# See what statements are available in a filing, organized by category xbrl = filing.xbrl() categories = xbrl.statements.get_statements_by_category() for category, stmts in categories.items(): for stmt in stmts: print(f"{category}: {stmt['type']} - {stmt.get('title', '')}")` Error Handling -------------- ### Common Errors and Solutions `from edgar.enums import validate_statement_type, StatementType # Typo in string try: validate_statement_type("income") except ValidationError as e: print(e) # "Did you mean: income_statement?" # Wrong type try: validate_statement_type(123) except TypeError as e: print(e) # "Statement must be StatementType or str" # Completely invalid try: validate_statement_type("invalid_statement") except ValidationError as e: print(e) # "Use StatementType enum for autocomplete..."` * * * Impact Summary -------------- **FEAT-005 delivers on EdgarTools principles:** * **Beginner-friendly**: Makes financial statement exploration discoverable * **Simple yet powerful**: Two APIs for different use cases * **Joyful UX**: IDE autocomplete and helpful error messages **Key improvements:** - IDE autocomplete for financial statement types - Enhanced validation with financial context - Clear separation between Company and XBRL APIs - Educational categorization of statement types - Full backwards compatibility maintained - Consistent design with FormType and PeriodType Back to top --- # Standardization - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/xbrl/concepts/standardization/#xbrl-standardization-concepts-reference) XBRL Standardization Concepts Reference ======================================= This document describes the 95 standard concepts used by EdgarTools to normalize XBRL financial data across different companies. These concepts enable cross-company comparison by mapping the ~18,000 possible SEC GAAP taxonomy tags to a consistent set of standardized line items. How Standardization Works ------------------------- **Labels are always preserved** - the company's original presentation is shown exactly as filed. Standardization adds a `standard_concept` column to DataFrames, mapping each line item to one of 95 standard categories: `# Get a statement DataFrame df = statement.to_dataframe() # Labels show original company presentation # standard_concept maps to standard categories for analysis print(df[['label', 'standard_concept']].head()) # label standard_concept # 0 Cash and cash items CashAndMarketableSecurities # 1 Trade receivables, net TradeReceivables # 2 Prepaid expenses OtherNonOperatingCurrentAssets # Aggregate by standard concept for cross-company comparison standardized = df.groupby('standard_concept')[['2024-09-30']].sum()` Overview -------- | Metric | Value | | --- | --- | | **Total Standard Concepts** | 95 | | **XBRL Tags Mapped** | 2,067 | | **Coverage** | ~95% of common financial statement tags | | **Source** | mpreiss9's production taxonomy (390+ companies) | Architecture ------------ `XBRL Tag (e.g., AccountsPayableCurrent) ↓ gaap_mappings.json (2,067 mappings) Standard Concept (e.g., TradePayables) ↓ display_names.json (95 mappings) Display Name (e.g., "Accounts Payable")` Industry Context ---------------- The number of standardized concepts varies by provider: | Provider | Line Items | Notes | | --- | --- | --- | | EdgarTools | 95 | Production-tested on 390+ companies | | Capital IQ | ~100-150 | Varies by data package | | Bloomberg | ~80-120 | Core financial line items | | Refinitiv | ~100-200 | Standardized fundamentals | | Morningstar | ~80-100 | Balance/Income/Cash Flow | * * * Standard Concepts by Statement ------------------------------ ### Balance Sheet - Current Assets | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `CashAndMarketableSecurities` | Cash and Cash Equivalents | 61 | `AssetBackedSecuritiesAtCarryingValue`, `AvailableForSaleDebtSecuritiesAmortizedCostBasis`, `AvailableForSaleSecurities`, ... | | `TradeReceivables` | Accounts Receivable | 34 | `AccountsAndNotesReceivableNet`, `AccountsAndOtherReceivablesNetCurrent`, `AccountsNotesAndLoansReceivableNetCurrent`, ... | | `Inventories` | Inventory | 66 | `AgriculturalRelatedInventory`, `AgriculturalRelatedInventoryFeedAndSupplies`, `AgriculturalRelatedInventoryGrowingCrops`, ... | | `DeferredTaxCurrentAssets` | Deferred Tax Assets, Current | 44 | `DeferredIncomeTaxesAndOtherAssetsCurrent`, `DeferredIncomeTaxesAndOtherTaxReceivableCurrent`, `DeferredTaxAssetsDeferredIncome`, ... | | `OtherOperatingCurrentAssets` | Other Current Assets | 50 | `AdvanceRoyaltiesCurrent`, `AdvancesOnInventoryPurchases`, `AmountOfDeferredCostsRelatedToLongTermContracts`, ... | | `OtherNonOperatingCurrentAssets` | Other Non-Operating Current Assets | 131 | `AccountsReceivableRelatedParties`, `AccountsReceivableRelatedPartiesCurrent`, `AllowanceForDoubtfulOtherReceivablesCurrent`, ... | | `RetirementRelatedCurrentAssets` | Retirement Related Assets, Current | 1 | `DefinedBenefitPlanCurrentAssets` | | `CurrentAssetsTotal` | Total Current Assets | 1 | `AssetsCurrent` | ### Balance Sheet - Non-Current Assets | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `PlantPropertyEquipmentNet` | Property, Plant and Equipment | 53 | `AccumulatedDepreciationDepletionAndAmortizationPropertyPlantAndEquipment`, `AcquisitionCostsCumulative`, `BuildingsAndImprovementsGross`, ... | | `Goodwill` | Goodwill | 10 | `Goodwill`, `GoodwillGross`, `GoodwillImpairedAccumulatedImpairmentLoss`, ... | | `IntangibleAssets` | Intangible Assets | 17 | `BusinessCombinationRecognizedIdentifiableAssetsAcquiredAndLiabilitiesAssumedIntangibles`, `FiniteLivedCustomerListsGross`, `FiniteLivedCustomerRelationshipsGross`, ... | | `LongtermInvestments` | Long-Term Investments | 60 | `AdvancesToAffiliate`, `AuctionRateSecuritiesNoncurrent`, `AvailableForSaleSecuritiesDebtMaturitiesAfterFiveThroughTenYearsFairValue`, ... | | `DeferredTaxNoncurrentAssets` | Deferred Tax Assets, Non-Current | 45 | `DeferredIncomeTaxAssetsNet`, `DeferredIncomeTaxesAndOtherAssetsNoncurrent`, `DeferredTaxAssetsCapitalLossCarryforwards`, ... | | `OtherOperatingNonCurrentAssets` | Other Non-Current Assets | 49 | `AccountsReceivableExcludingAccruedInterestAfterAllowanceForCreditLossNoncurrent`, `AccountsReceivableGross`, `AccountsReceivableGrossNoncurrent`, ... | | `OtherNonOperatingNonCurrentAssets` | Other Non-Operating Non-Current Assets | 140 | `AccountsReceivableRelatedParties`, `AccountsReceivableRelatedPartiesNoncurrent`, `AccruedFeesAndOtherRevenueReceivable`, ... | | `OperatingLeaseRightOfUseAsset` | Operating Lease Right-of-Use Asset | 1 | `OperatingLeaseRightOfUseAsset` | | `RetirementRelatedNonCurrentAssets` | Retirement Related Assets, Non-Current | 4 | `AssetRetirementObligationLegallyRestrictedAssetsFairValue`, `DeferredCompensationPlanAssets`, `DefinedBenefitPlanAmountsRecognizedInBalanceSheet`, ... | | `NonCurrentAssetsTotal` | Total Non-Current Assets | 2 | `AssetsNoncurrent`, `NoncurrentAssets` | | `Assets` | Total Assets | 2 | `Assets`, `AssetsNet` | ### Balance Sheet - Current Liabilities | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `TradePayables` | Accounts Payable | 27 | `AccountsPayableAndAccruedLiabilitiesCurrent`, `AccountsPayableAndAccruedLiabilitiesCurrentAndNoncurrent`, `AccountsPayableAndOtherAccruedLiabilities`, ... | | `ShortTermDebt` | Short-Term Debt | 62 | `BankLoans`, `BankOverdrafts`, `BorrowingsUnderGuaranteedInvestmentAgreements`, ... | | `DeferredTaxCurrentLiabilities` | Deferred Tax Liabilities, Current | 20 | `DeferredIncomeTaxLiabilities`, `DeferredIncomeTaxLiabilitiesNet`, `DeferredTaxAssetsLiabilitiesNetCurrent`, ... | | `TaxesPayable` | Taxes Payable | 7 | `AccrualForTaxesOtherThanIncomeTaxesCurrent`, `AccruedIncomeTaxes`, `AccruedIncomeTaxesCurrent`, ... | | `DividendsPayable` | Dividends Payable | 2 | `DividendsPayableCurrent`, `DividendsPayableCurrentAndNoncurrent` | | `OtherOperatingCurrentLiabilities` | Other Current Liabilities | 68 | `AccrualForEnvironmentalLossContingencies`, `AccruedAdvertisingCurrent`, `AccruedAdvertisingCurrentAndNoncurrent`, ... | | `OtherNonOperatingCurrentLiabilities` | Other Non-Operating Current Liabilities | 82 | `AccountsPayableOtherCurrentAndNoncurrent`, `AccrualForTaxesOtherThanIncomeTaxesCurrentAndNoncurrent`, `AccruedCappingClosurePostClosureAndEnvironmentalCosts`, ... | | `RetirementRelatedCurrentLiabilities` | Retirement Related Liabilities, Current | 18 | `DeferredCompensationCashBasedArrangementsLiabilityCurrent`, `DeferredCompensationLiabilityCurrent`, `DeferredCompensationLiabilityCurrentAndNoncurrent`, ... | | `OperatingLeaseCurrentDebtEquivalent` | Operating Lease Liability, Current | 2 | `OperatingLeaseLiability`, `OperatingLeaseLiabilityCurrent` | | `CurrentLiabilitiesTotal` | Total Current Liabilities | 1 | `LiabilitiesCurrent` | ### Balance Sheet - Non-Current Liabilities | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `LongTermDebt` | Long-Term Debt | 71 | `CapitalLeaseObligations`, `CapitalLeaseObligationsNoncurrent`, `CommercialPaperNoncurrent`, ... | | `DeferredTaxNonCurrentLiabilities` | Deferred Tax Liabilities, Non-Current | 35 | `AccruedIncomeTaxes`, `AccruedIncomeTaxesNoncurrent`, `AccumulatedDeferredInvestmentTaxCredit`, ... | | `OtherOperatingNonCurrentLiabilities` | Other Non-Current Liabilities | 31 | `AccountsPayableAndAccruedLiabilitiesCurrentAndNoncurrent`, `AccountsPayableAndAccruedLiabilitiesNoncurrent`, `AccountsPayableAndOtherAccruedLiabilities`, ... | | `OtherNonOperatingNonCurrentLiabilities` | Other Non-Operating Non-Current Liabilities | 81 | `AccountsPayableOtherCurrentAndNoncurrent`, `AccrualForTaxesOtherThanIncomeTaxesCurrentAndNoncurrent`, `AccruedEmployeeBenefitsCurrentAndNoncurrent`, ... | | `RetirementRelatedNonCurrentLiabilities` | Retirement Related Liabilities, Non-Current | 18 | `AssetRetirementObligation`, `DeferredCompensationLiabilityClassifiedNoncurrent`, `DeferredCompensationLiabilityCurrentAndNoncurrent`, ... | | `OperatingLeaseNonCurrentDebtEquivalent` | Operating Lease Liability, Non-Current | 3 | `OperatingLeaseLiability`, `OperatingLeaseLiabilityNoncurrent`, `OperatingLeaseLiabilityStatementOfFinancialPositionExtensibleList` | | `OngoingOperatingProvisions(WarrantiesEtc)` | Warranty and Other Provisions | 22 | `ContractWithCustomerRefundLiability`, `ContractWithCustomerRefundLiabilityNoncurrent`, `CustomerAdvancesAndDeposits`, ... | | `DefinteLivedOperatingProvisions(DecommissioningEtc)` | Asset Retirement Obligations | 10 | `AccruedCappingClosurePostClosureAndEnvironmentalCosts`, `AccruedCappingClosurePostClosureAndEnvironmentalCostsNoncurrent`, `AssetRetirementObligationsNoncurrent`, ... | | `RestructuringProvisions` | Restructuring Provisions | 2 | `RestructuringReserve`, `RestructuringReserveNoncurrent` | | `NonCurrentLiabilitiesTotal` | Total Non-Current Liabilities | 1 | `LiabilitiesNoncurrent` | | `Liabilities` | Total Liabilities | 1 | `Liabilities` | ### Balance Sheet - Equity | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `CommonEquity` | Total Stockholders' Equity | 50 | `AccumulatedOtherComprehensiveIncomeLossAvailableForSaleSecuritiesAdjustmentNetOfTax`, `AccumulatedOtherComprehensiveIncomeLossCumulativeChangesInNetGainLossFromCashFlowHedgesEffectNetOfTax`, `AccumulatedOtherComprehensiveIncomeLossDefinedBenefitPensionAndOtherPostretirementPlansNetOfTax`, ... | | `PreferredStock` | Preferred Stock | 6 | `AdditionalPaidInCapitalPreferredStock`, `PreferredStockRedemptionAmount`, `PreferredStockSharesSubscribedButUnissuedSubscriptionsReceivable`, ... | | `TreasuryShares` | Treasury Stock | 2 | `TreasuryStockCommonShares`, `TreasuryStockShares` | | `MinorityInterestBalance` | Noncontrolling Interest | 10 | `MembersEquityAttributableToNoncontrollingInterest`, `MinorityInterest`, `MinorityInterestInJointVentures`, ... | | `TemporaryAndMezzanineFinancing` | Temporary Equity | 11 | `RedeemableNoncontrollingInterestEquityCarryingAmount`, `RedeemableNoncontrollingInterestEquityCommonCarryingAmount`, `RedeemableNoncontrollingInterestEquityCommonFairValue`, ... | | `AllEquityBalance` | Total Equity | 1 | `StockholdersEquity` | | `AllEquityBalanceIncludingMinorityInterest` | Total Equity Including Noncontrolling Interest | 4 | `AociIncludingPortionAttributableToNoncontrollingInterestTax`, `DefinedBenefitPlanAccumulatedOtherComprehensiveIncomeBeforeTax`, `LimitedLiabilityCompanyLlcMembersEquityIncludingPortionAttributableToNoncontrollingInterest`, ... | | `LiabilitiesAndEquity` | Total Liabilities and Equity | 1 | `LiabilitiesAndStockholdersEquity` | ### Income Statement - Revenue & Gross Profit | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `Revenue` | Revenue | 139 | `AdmissionsRevenue`, `AdvertisingRevenue`, `BinderSalesRevenue`, ... | | `CostOfGoodsAndServicesSold` | Cost of Revenue | 142 | `AdvertisingRevenueCost`, `AffiliateCosts`, `AircraftRentalAndLandingFees`, ... | | `GrossProfit` | Gross Profit | 1 | `GrossProfit` | ### Income Statement - Operating Expenses | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `ResearchAndDevelopementExpenses` | Research and Development Expense | 6 | `ExplorationExpense`, `ResearchAndDevelopmentAssetAcquiredOtherThanThroughBusinessCombinationWrittenOff`, `ResearchAndDevelopmentExpense`, ... | | `SellingGeneralAndAdminExpenses` | Selling, General and Administrative Expense | 16 | `CommunicationsAndInformationTechnology`, `GeneralAndAdministrativeExpense`, `GeneralInsuranceExpense`, ... | | `MarketingExpenses` | Marketing Expense | 4 | `AdvertisingExpense`, `CooperativeAdvertisingExpense`, `MarketingAndAdvertisingExpense`, ... | | `DepreciationExpense` | Depreciation Expense | 13 | `CapitalizedComputerSoftwareAmortization`, `CapitalizedComputerSoftwareImpairments`, `CostDepreciationAmortizationAndDepletion`, ... | | `AmortizationOfIntangibles` | Amortization of Intangibles | 4 | `AmortizationOfIntangibleAssets`, `ImpairmentOfIntangibleAssetsExcludingGoodwill`, `ImpairmentOfIntangibleAssetsFinitelived`, ... | | `OtherOperatingExpense` | Other Operating Expense | 45 | `AccretionExpense`, `AcquisitionCosts`, `AllocatedShareBasedCompensationExpense`, ... | | `RestructuringExpenseBenefit` | Restructuring Expense | 32 | `AmortizationOfAcquisitionCosts`, `BusinessCombinationAcquisitionRelatedCosts`, `BusinessCombinationIntegrationRelatedCosts`, ... | | `GoodwillWriteoffs` | Goodwill Impairment | 9 | `AdjustmentForAmortization`, `AssetImpairmentCharges`, `CostOfGoodsAndServicesSoldAmortization`, ... | | `CostsSubtotal` | Total Costs and Expenses | 5 | `BenefitsLossesAndExpenses`, `CostsAndExpenses`, `EmployeeBenefitsAndShareBasedCompensation`, ... | | `OperatingIncomeLoss` | Operating Income | 1 | `OperatingIncomeLoss` | ### Income Statement - Non-Operating & Tax | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `InterestExpense` | Interest Expense | 38 | `AmortizationOfDebtDiscountPremium`, `AmortizationOfDeferredHedgeGains`, `AmortizationOfFinancingCosts`, ... | | `InterestIncome` | Interest Income | 20 | `InterestAndDividendIncomeOperating`, `InterestAndOtherIncome`, `InterestIncomeExpenseAfterProvisionForLoanLoss`, ... | | `NonoperatingIncomeExpense` | Non-Operating Income (Expense) | 199 | `AccretionAmortizationOfDiscountsAndPremiumsInvestments`, `AccretionExpenseIncludingAssetRetirementObligations`, `AvailableForSaleSecuritiesGrossRealizedGainLossNet`, ... | | `SpecialItemsIncomeExpense(Pretax)` | Special Items | 2 | `UnusualOrInfrequentItemInsuranceProceeds`, `UnusualOrInfrequentItemNetOfInsuranceProceeds` | | `PretaxIncomeLoss` | Income Before Tax | 1 | `IncomeLossFromContinuingOperationsBeforeIncomeTaxesExtraordinaryItemsNoncontrollingInterest` | | `IncomeTaxes` | Income Tax Expense | 78 | `AdjustmentsToAdditionalPaidInCapitalTaxEffectFromShareBasedCompensation`, `CurrentFederalStateAndLocalTaxExpenseBenefit`, `CurrentFederalTaxExpenseBenefit`, ... | | `IncomeLossContinuingOperations` | Income from Continuing Operations | 1 | `IncomeLossFromContinuingOperations` | | `ExtraordinaryItemsIncomeExpense(PostTax)` | Extraordinary Items | 35 | `DiscontinuedOperationAmountOfAdjustmentToPriorPeriodGainLossOnDisposalBeforeIncomeTax`, `DiscontinuedOperationAmountOfAdjustmentToPriorPeriodGainLossOnDisposalNetOfTax`, `DiscontinuedOperationAmountOfOtherIncomeLossFromDispositionOfDiscontinuedOperationNetOfTax`, ... | | `MinorityInterestIncomeExpense` | Net Income Attributable to Noncontrolling Interest | 10 | `ComprehensiveIncomeNetOfTaxAttributableToNoncontrollingInterest`, `EquityMethodInvestmentOtherThanTemporaryImpairment`, `IncomeLossFromContinuingOperationsAttributableToNoncontrollingEntity`, ... | | `NetIncome` | Net Income | 2 | `IncomeLossAttributableToParent`, `NetIncomeLoss` | | `NetIncomeToCommonShareholders` | Net Income to Common Shareholders | 3 | `NetIncomeLossAvailableToCommonStockholdersBasic`, `NetIncomeLossFromContinuingOperationsAvailableToCommonShareholdersBasic`, `ParticipatingSecuritiesDistributedAndUndistributedEarnings` | | `PreferredDividendExpense` | Preferred Dividends | 13 | `DividendsPreferredStock`, `DividendsPreferredStockCash`, `DividendsPreferredStockStock`, ... | | `ProfitLoss` | Profit or Loss | 1 | `ProfitLoss` | ### Cash Flow & Capital | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `CapitalExpenses` | Capital Expenditures | 4 | `PaymentsToAcquireOtherProductiveAssets`, `PaymentsToAcquireOtherPropertyPlantAndEquipment`, `PaymentsToAcquireProductiveAssets`, ... | | `CommonDividendsPaid` | Dividends Paid | 4 | `Dividends`, `DividendsCash`, `DividendsCommonStock`, ... | | `EquityExpenseIncome(BuybackIssued)` | Stock Repurchases (Issuances) | 4 | `PaymentsForRepurchaseOfCommonStock`, `ProceedsFromIssuanceOfCommonStock`, `ProceedsFromSaleOfTreasuryStock`, ... | ### Per Share Data | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `CommonDividendsPerShare` | Dividends Per Share | 3 | `CommonStockDividendsPerShareCashPaid`, `CommonStockDividendsPerShareDeclared`, `DividendsPayableAmountPerShare` | | `SharesAverage` | Weighted Average Shares Outstanding | 2 | `WeightedAverageBasicSharesOutstandingProForma`, `WeightedAverageNumberOfSharesOutstandingBasic` | | `SharesDilutionAdjustment` | Dilution Adjustment | 2 | `IncrementalCommonSharesAttributableToShareBasedPaymentArrangements`, `WeightedAverageNumberDilutedSharesOutstandingAdjustment` | | `SharesFullyDilutedAverage` | Weighted Average Shares Outstanding, Diluted | 2 | `WeightedAverageNumberOfDilutedSharesOutstanding`, `WeightedAverageNumberOfShareOutstandingBasicAndDiluted` | | `SharesIssued` | Shares Issued | 2 | `CommonStockSharesIssued`, `SharesIssued` | | `SharesYearEnd` | Shares Outstanding | 3 | `CommonStockSharesOutstanding`, `EntityCommonStockSharesOutstanding`, `SharesOutstanding` | ### Operating Lease Commitments | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `OperatingLeaseCommitmentYear1` | Operating Lease Commitment, Year 1 | 1 | `OperatingLeasesFutureMinimumPaymentsDueCurrent` | | `OperatingLeaseCommitmentYear2` | Operating Lease Commitment, Year 2 | 1 | `OperatingLeasesFutureMinimumPaymentsDueInTwoYears` | | `OperatingLeaseCommitmentYear3` | Operating Lease Commitment, Year 3 | 1 | `OperatingLeasesFutureMinimumPaymentsDueInThreeYears` | | `OperatingLeaseCommitmentYear4` | Operating Lease Commitment, Year 4 | 1 | `OperatingLeasesFutureMinimumPaymentsDueInFourYears` | | `OperatingLeaseCommitmentYear5` | Operating Lease Commitment, Year 5 | 1 | `OperatingLeasesFutureMinimumPaymentsDueInFiveYears` | | `OperatingLeaseCommitmentAfterYear5` | Operating Lease Commitment, Thereafter | 1 | `OperatingLeasesFutureMinimumPaymentsDueThereafter` | ### Intangible Amortization Forecast | Standard Concept | Display Name | XBRL Tags | Examples | | --- | --- | --- | --- | | `ForecastedIntangibleAmortizationYear1` | Forecasted Amortization, Year 1 | 3 | `FiniteLivedIntangibleAssetsAmortizationExpenseNextRollingTwelveMonths`, `FiniteLivedIntangibleAssetsAmortizationExpenseNextTwelveMonths`, `FiniteLivedIntangibleAssetsAmortizationExpenseRemainderOfFiscalYear` | | `ForecastedIntangibleAmortizationYear2` | Forecasted Amortization, Year 2 | 2 | `FiniteLivedIntangibleAssetsAmortizationExpenseRollingYearTwo`, `FiniteLivedIntangibleAssetsAmortizationExpenseYearTwo` | | `ForecastedIntangibleAmortizationYear3` | Forecasted Amortization, Year 3 | 2 | `FiniteLivedIntangibleAssetsAmortizationExpenseRollingYearThree`, `FiniteLivedIntangibleAssetsAmortizationExpenseYearThree` | | `ForecastedIntangibleAmortizationYear4` | Forecasted Amortization, Year 4 | 1 | `FiniteLivedIntangibleAssetsAmortizationExpenseYearFour` | | `ForecastedIntangibleAmortizationYear5` | Forecasted Amortization, Year 5 | 1 | `FiniteLivedIntangibleAssetsAmortizationExpenseYearFive` | | `ForecastedIntangibleAmortizationAfterYear5` | Forecasted Amortization, Thereafter | 2 | `FiniteLivedIntangibleAssetsAmortizationExpenseAfterYearFive`, `FiniteLivedIntangibleAssetsAmortizationExpenseRollingAfterYearFive` | * * * Concept Details --------------- ### Balance Sheet Concepts #### Current vs Non-Current Classification The taxonomy distinguishes between current (due within 1 year) and non-current items: * **Current Assets**: `CashAndMarketableSecurities`, `TradeReceivables`, `Inventories`, etc. * **Non-Current Assets**: `PlantPropertyEquipmentNet`, `Goodwill`, `LongtermInvestments`, etc. * **Current Liabilities**: `TradePayables`, `ShortTermDebt`, etc. * **Non-Current Liabilities**: `LongTermDebt`, `DeferredTaxNonCurrentLiabilities`, etc. #### Operating vs Non-Operating Classification Items are also classified by their relationship to core operations: * **Operating**: Related to primary business activities (e.g., `OtherOperatingCurrentAssets`) * **Non-Operating**: Related to financing/investing activities (e.g., `OtherNonOperatingCurrentAssets`) ### Income Statement Concepts #### Revenue Recognition All revenue-related XBRL tags (139 variations) map to the single `Revenue` concept. This includes: - `RevenueFromContractWithCustomerExcludingAssessedTax` - `Revenues` - `SalesRevenueNet` - `SalesRevenueGoodsNet` - And 135 more industry-specific variations #### Cost of Revenue Cost tags (142 variations) map to `CostOfGoodsAndServicesSold`: - `CostOfRevenue` - `CostOfGoodsAndServicesSold` - `CostOfGoodsSold` - `CostOfServices` ### Operating Lease Accounting (ASC 842) Following the ASC 842 lease accounting standard, the taxonomy includes: * **Right-of-Use Asset**: `OperatingLeaseRightOfUseAsset` * **Current Liability**: `OperatingLeaseCurrentDebtEquivalent` * **Non-Current Liability**: `OperatingLeaseNonCurrentDebtEquivalent` * **Future Commitments**: Years 1-5 and thereafter ### Provisions and Reserves The taxonomy distinguishes between: * **Ongoing Provisions**: `OngoingOperatingProvisions(WarrantiesEtc)` - recurring obligations * **Definite-Lived Provisions**: `DefinteLivedOperatingProvisions(DecommissioningEtc)` - asset retirement * **Restructuring**: `RestructuringProvisions` - one-time reorganization costs * * * Ambiguous Tags -------------- Some XBRL tags can map to multiple concepts depending on context. These are flagged as "ambiguous" and require context-aware resolution (Phase 4 of implementation). **Total Ambiguous Tags**: 215 (9.2% of mapped tags) ### Common Ambiguity Types 1. **Current/Non-Current Ambiguity** (202 tags) 2. Example: `AccountsPayableCurrentAndNoncurrent` → `TradePayables` OR `OtherOperatingNonCurrentLiabilities` 3. Resolution: Based on balance sheet section placement 4. **Asset/Liability Ambiguity** (12 tags) 5. Example: `DeferredTaxAssetsLiabilitiesNet` → `DeferredTaxNoncurrentAssets` OR `DeferredTaxNonCurrentLiabilities` 6. Resolution: Based on sign (positive = asset, negative = liability) 7. **Operating/Non-Operating Ambiguity** 8. Example: `OtherAssetsNoncurrent` → `OtherOperatingNonCurrentAssets` OR `OtherNonOperatingNonCurrentAssets` 9. Resolution: Based on statement section context * * * Excluded Tags (DropThisItem) ---------------------------- 276 XBRL tags are explicitly excluded from standardization because they: - Confuse financial analysis (EPS details, pro-forma metrics) - Are redundant with other tags - Don't map cleanly to standard concepts **Examples of excluded tags:** - `AcceleratedShareRepurchasesFinalPricePaidPerShare` - `BasicEarningsPerShareProForma` - `BusinessAcquisitionProFormaEarningsPerShareBasic` - Various per-share calculation details * * * Deprecated Tags --------------- 410 XBRL tags are marked as deprecated by the SEC with the year of deprecation. The mapping still works for historical filings, but these tags should not appear in recent filings. **Example:** - `Revenues` (deprecated 2018) → Still maps to `Revenue` - `AccountsPayableRelatedPartiesCurrent` (deprecated 2023) → Still maps to `TradePayables` * * * Python API (v5.9.0+) -------------------- EdgarTools provides several APIs for working with standardized concepts: ### Module-Level Singletons (Recommended) For best performance, use the module-level singletons which load mappings once per session: `from edgar.xbrl.standardization import ( get_default_mapper, get_default_store, StandardConcept, StandardizationCache ) # Get the singleton mapper - eliminates redundant file I/O mapper = get_default_mapper() # Map a company concept to standardized label label = mapper.map_concept( "us-gaap_AccountsPayableCurrent", "Accounts Payable", {"statement_type": "BalanceSheet"} ) # Returns: "Accounts Payable"` ### StandardConcept Enum Type-safe enum for all standardized concepts: `from edgar.xbrl.standardization import StandardConcept # Access standard concept labels StandardConcept.REVENUE.value # "Revenue" StandardConcept.NET_INCOME.value # "Net Income" StandardConcept.TOTAL_ASSETS.value # "Total Assets" StandardConcept.ACCOUNTS_PAYABLE.value # "Accounts Payable" # Look up a concept by its label concept = StandardConcept.get_from_label("Revenue") # Returns: StandardConcept.REVENUE # Get all available standard values all_values = StandardConcept.get_all_values() # Returns: {'Revenue', 'Net Income', 'Total Assets', ...}` ### StandardizationCache (Per-XBRL Caching) For high-performance workflows, use `StandardizationCache` which caches results per XBRL instance: `# The cache is automatically attached to XBRL instances xbrl = filing.xbrl() # Access via the standardization property cache = xbrl.standardization # Get cached label lookups label = cache.get_standard_label( "us-gaap_Revenue", "Total Revenue", {"statement_type": "IncomeStatement"} ) # Standardize entire statement with caching raw_data = xbrl.get_statement_data("IncomeStatement") standardized = cache.standardize_statement_data(raw_data, "IncomeStatement") # Check cache statistics print(cache.cache_stats) # {'label_cache_size': 42, 'statement_cache_size': 1, 'cached_statements': ['IncomeStatement']} # Clear cache when needed cache.clear_cache() # Clear all cache.clear_cache("IncomeStatement") # Clear specific statement` ### Reverse Index API (Low-Level) Direct O(1) lookup for XBRL tags: `from edgar.xbrl.standardization.reverse_index import ( get_standard_concept, get_display_name, lookup ) # Simple lookup concept = get_standard_concept("AccountsPayableCurrent") # Returns: "TradePayables" display = get_display_name("AccountsPayableCurrent") # Returns: "Accounts Payable" # Full result with metadata result = lookup("AccountsPayableCurrentAndNoncurrent") result.is_ambiguous # True result.standard_concepts # ['TradePayables', 'OtherOperatingNonCurrentLiabilities'] result.display_names # ['Accounts Payable', 'Other Non-Current Liabilities'] result.comment # 'Curr/NonCurr ambiguity'` ### Checking Coverage `from edgar.xbrl.standardization.reverse_index import get_reverse_index index = get_reverse_index() print(index.stats) # {'total_mappings': 2067, 'ambiguous_count': 215, 'deprecated_count': 410, 'excluded_count': 276}` * * * Context-Aware Disambiguation ---------------------------- Starting in v5.9.0, EdgarTools uses context-aware disambiguation to resolve ambiguous tags. ### How It Works Ambiguous tags (e.g., `AccountsPayableCurrentAndNoncurrent`) can map to multiple concepts. EdgarTools uses two complementary strategies to disambiguate: #### 1\. Calculation Parent Derivation When an item has a `calculation_parent`, EdgarTools infers its section: `# Item with calculation_parent="us-gaap:AssetsCurrent" # → Inferred section: "Current Assets" # → Resolves ambiguity toward current asset concepts # Supported parent → section mappings: # AssetsCurrent → "Current Assets" # AssetsNoncurrent → "Non-Current Assets" # LiabilitiesCurrent → "Current Liabilities" # LiabilitiesNoncurrent → "Non-Current Liabilities" # StockholdersEquity → "Equity"` #### 2\. Bottom-Up Section Scanning (mpreiss9 method) For items without calculation parents, EdgarTools scans the statement from bottom to top, using subtotals as section boundaries: `Total Current Assets ← Defines boundary for "Current Assets" section ↑ All items above until next subtotal belong to "Current Assets" ↑ Property, Plant & Equipment ← This item gets assigned "Current Assets" section based on its position relative to subtotal` ### Passing Context to the Mapper When calling the mapper directly, you can provide context for disambiguation: `mapper = get_default_mapper() # Provide context for accurate disambiguation label = mapper.map_concept( "us-gaap_OtherAssetsNoncurrent", "Other Assets", { "statement_type": "BalanceSheet", "section": "Non-Current Assets", # Helps resolve operating vs non-operating "calculation_parent": "us-gaap:AssetsNoncurrent", "level": 2, "is_total": False } )` ### Context Keys | Key | Description | Used For | | --- | --- | --- | | `statement_type` | "BalanceSheet", "IncomeStatement", etc. | Statement-specific matching | | `section` | "Current Assets", "Equity", etc. | Disambiguating current/non-current | | `calculation_parent` | Parent concept in calculation tree | Deriving section automatically | | `level` | Indentation level (0-5) | Identifying subtotals vs details | | `is_total` | True for subtotal/total rows | Section boundary detection | | `balance` | "debit" or "credit" | Sign-based disambiguation | * * * Files ----- | File | Purpose | | --- | --- | | `gaap_mappings.json` | 2,067 XBRL tag → standard concept mappings | | `display_names.json` | 95 standard concept → display name mappings | | `exclusions.py` | 276 excluded (DropThisItem) tags | | `reverse_index.py` | O(1) lookup implementation | | `core.py` | StandardConcept enum, MappingStore, ConceptMapper, standardize\_statement | | `cache.py` | StandardizationCache for per-XBRL instance caching | | `sections.py` | Section classification for disambiguation | | `__init__.py` | Module exports and singleton accessors | * * * Version History --------------- | Version | Date | Changes | | --- | --- | --- | | 5.9.0 | 2026-01 | Context-aware disambiguation, bottom-up section scanning, StandardizationCache | | 5.8.0 | 2026-01 | Module-level singletons (get\_default\_mapper, get\_default\_store) | | 1.0 | 2026-01 | Initial release with 95 concepts, 2,067 mappings | * * * Related Resources ----------------- * [XBRL Documentation Hub](https://edgartools.readthedocs.io/en/stable/xbrl/) - Central navigation for all XBRL docs * [Extract Financial Statements Guide](https://edgartools.readthedocs.io/en/stable/guides/extract-statements/) - Complete guide to extracting financial data * [Dimension Handling Guide](https://edgartools.readthedocs.io/en/stable/xbrl/concepts/dimension-handling/) - Understanding dimensional data and segment breakdowns * [Multi-Period Analysis Guide](https://edgartools.readthedocs.io/en/stable/xbrl/guides/multi-period-analysis/) - Working with XBRLS for multi-period comparison * * * Need help scaling XBRL standardization? EdgarTools maps 2,067 XBRL tags to 95 standard concepts. But production pipelines hit custom extensions, deprecated tags, and taxonomy version changes that go beyond standard mappings. * **[XBRL consulting for AI & data teams →](https://www.edgar.tools/consulting/xbrl?utm_source=edgartools-docs&utm_medium=see-live&utm_content=xbrl-standardization) ** * **[See all SEC data consulting services →](https://www.edgar.tools/consulting?utm_source=edgartools-docs&utm_medium=see-live&utm_content=xbrl-standardization) ** From the creator of edgartools. [Book a call →](https://www.edgar.tools/consulting/xbrl?utm_source=edgartools-docs&utm_medium=see-live&utm_content=xbrl-standardization#contact) Credits ------- The standardization taxonomy is based on the production mappings shared by [@mpreiss9](https://github.com/mpreiss9) , tested across 390+ companies. See: [GitHub Issue #494](https://github.com/dgunning/edgartools/issues/494) Back to top --- # XBRL - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/api/xbrl/#xbrl-api-reference) XBRL API Reference ================== The XBRL module provides comprehensive parsing and processing of XBRL (eXtensible Business Reporting Language) data from SEC filings. It includes support for statement standardization, multi-period analysis, and advanced querying capabilities. Module Overview --------------- The XBRL module is organized into several key components: * **Core Classes**: `XBRL`, `XBRLS` for parsing and managing XBRL documents * **Statement Processing**: `Statements`, `Statement` for working with financial statements * **Facts Querying**: `FactsView`, `FactQuery` for querying XBRL facts * **Multi-Period Analysis**: `StitchedStatements`, `StitchedStatement` for comparative analysis * **Standardization**: `StandardConcept` for normalizing company-specific concepts * **Rendering**: `RenderedStatement` for formatted output Core Classes ------------ ### XBRL The main class for parsing and working with XBRL documents from SEC filings. `from edgar.xbrl import XBRL class XBRL: """Main XBRL parser integrating all components of the XBRL parsing system."""` #### Factory Methods #### from\_filing() `@classmethod def from_filing(cls, filing: Filing) -> XBRL` Create an XBRL instance from a Filing object. **Parameters:** - `filing`: SEC filing object containing XBRL data **Returns:** `XBRL` instance **Example:** `from edgar import Company from edgar.xbrl import XBRL company = Company("AAPL") filing = company.latest("10-K") xbrl = XBRL.from_filing(filing)` #### from\_directory() `@classmethod def from_directory(cls, directory: str) -> XBRL` Create an XBRL instance from a directory containing XBRL files. **Parameters:** - `directory`: Path to directory containing XBRL files **Returns:** `XBRL` instance #### from\_files() `@classmethod def from_files(cls, files: List[str]) -> XBRL` Create an XBRL instance from a list of XBRL files. **Parameters:** - `files`: List of file paths to XBRL documents **Returns:** `XBRL` instance #### Core Properties #### statements `@property def statements(self) -> Statements` Access to all financial statements in the XBRL document. **Returns:** `Statements` object for accessing individual statements **Example:** `# Access different statement types balance_sheet = xbrl.statements.balance_sheet() income_statement = xbrl.statements.income_statement() cash_flow = xbrl.statements.cash_flow_statement()` #### facts `@property def facts(self) -> FactsView` Access to all XBRL facts with querying capabilities. **Returns:** `FactsView` object for querying facts **Example:** `# Query facts by concept revenue_facts = xbrl.facts.by_concept("Revenue") # Convert to DataFrame for analysis facts_df = xbrl.facts.to_dataframe()` #### Statement Methods #### get\_statement() `def get_statement(self, statement_type: str) -> Optional[Statement]` Get a specific financial statement by type. **Parameters:** - `statement_type`: Statement type ("BalanceSheet", "IncomeStatement", "CashFlowStatement", etc.) **Returns:** `Statement` object or None if not found #### render\_statement() `def render_statement(self, statement_type: str, **kwargs) -> RenderedStatement` Render a financial statement with rich formatting. **Parameters:** - `statement_type`: Statement type to render - `**kwargs`: Additional rendering options **Returns:** `RenderedStatement` object **Example:** `# Render balance sheet rendered = xbrl.render_statement("BalanceSheet") print(rendered) # Render with custom options rendered = xbrl.render_statement("IncomeStatement", show_percentages=True, max_rows=50)` #### Data Conversion #### to\_pandas() `def to_pandas(self) -> pd.DataFrame` Convert XBRL facts to a pandas DataFrame. **Returns:** DataFrame with all facts and their attributes **Example:** `# Convert to DataFrame for analysis df = xbrl.to_pandas() print(df.columns) # ['concept', 'value', 'period', 'label', ...] # Filter for specific concepts revenue_df = df[df['concept'].str.contains('Revenue', case=False)]` ### XBRLS Container class for managing multiple XBRL documents for multi-period analysis. `from edgar.xbrl import XBRLS class XBRLS: """Container for multiple XBRL objects enabling multi-period analysis."""` #### Factory Methods #### from\_filings() `@classmethod def from_filings(cls, filings: List[Filing]) -> XBRLS` Create an XBRLS instance from multiple filings. **Parameters:** - `filings`: List of Filing objects **Returns:** `XBRLS` instance **Example:** `from edgar import Company from edgar.xbrl import XBRLS company = Company("AAPL") filings = company.get_filings(form="10-K").head(3) # Get 3 years xbrls = XBRLS.from_filings(filings)` #### Properties #### statements `@property def statements(self) -> StitchedStatements` Access to stitched statements showing multi-period data. **Returns:** `StitchedStatements` object **Example:** `# Get multi-period statements income_stmt = xbrls.statements.income_statement() balance_sheet = xbrls.statements.balance_sheet() # Render multi-period view print(income_stmt.render())` Statement Classes ----------------- ### Statements High-level interface for accessing financial statements from a single XBRL document. `class Statements: """High-level interface to all statements in an XBRL document."""` #### Statement Access Methods #### balance\_sheet() `def balance_sheet(self) -> Optional[Statement]` Get the balance sheet statement. **Returns:** `Statement` object or None #### income\_statement() `def income_statement(self) -> Optional[Statement]` Get the income statement. **Returns:** `Statement` object or None #### cash\_flow\_statement() `def cash_flow_statement(self) -> Optional[Statement]` Get the cash flow statement. **Returns:** `Statement` object or None #### statement\_of\_equity() `def statement_of_equity(self) -> Optional[Statement]` Get the statement of equity. **Returns:** `Statement` object or None #### comprehensive\_income() `def comprehensive_income(self) -> Optional[Statement]` Get the comprehensive income statement. **Returns:** `Statement` object or None **Example:** `statements = xbrl.statements # Access different statement types if statements.balance_sheet(): bs = statements.balance_sheet() print(f"Total Assets: {bs.get_concept_value('Assets')}") if statements.income_statement(): is_stmt = statements.income_statement() print(f"Revenue: {is_stmt.get_concept_value('Revenue')}")` ### Statement Individual financial statement with analysis and rendering capabilities. `class Statement: """A single financial statement extracted from XBRL data."""` #### Core Methods #### render() `def render(self, **kwargs) -> RenderedStatement` Render the statement with rich formatting. **Parameters:** - `**kwargs`: Rendering options (show\_percentages, max\_rows, etc.) **Returns:** `RenderedStatement` object #### to\_dataframe() `def to_dataframe( self, include_dimensions: bool = True, include_unit: bool = False, include_point_in_time: bool = False, presentation: bool = False ) -> pd.DataFrame` Convert statement to pandas DataFrame with optional transformations. **Parameters:** - `include_dimensions`: Include dimensional breakdowns (default: True) - `include_unit`: Include unit column (USD, shares, etc.) (default: False) - `include_point_in_time`: Include point-in-time column for instant facts (default: False) - `presentation`: Apply HTML-matching transformations using preferred\_sign (default: False) - False (default): Raw instance values from XML - True: Transform values to match SEC filing HTML display **Returns:** DataFrame with the following columns: - **Core columns**: `concept`, `label`, period columns (dates) - **Metadata columns** (always included): - `balance` — debit or credit (from XBRL taxonomy) - `weight` — calculation tree weight (+1 or -1) - `preferred_sign` — how the value should be displayed (from presentation linkbase) - `level` — nesting depth in the presentation tree (0=root, 1=section header, 2=line item, etc.) - `abstract` — True if this row is a section header, not a data row - `parent_concept` — calculation tree parent (the metric concept this rolls up to for summation math) - `parent_abstract_concept` — presentation tree parent (the section header this appears under for display hierarchy) - **Optional columns**: `dimension`, `unit`, `point_in_time` **Value Modes:** - **Raw mode** (default): Preserves values exactly as reported in instance document - **Presentation mode** (`presentation=True`): Applies transformations to match SEC HTML rendering - Cash Flow: outflows with preferred\_sign=-1 shown as negative - Income Statement: applies preferred\_sign transformations **Example:** `statement = xbrl.statements.income_statement() # Raw values (default) df_raw = statement.to_dataframe() # Returns actual XML values + metadata columns # Presentation mode (matches SEC HTML) df_presentation = statement.to_dataframe(presentation=True) # Returns transformed values matching 10-K HTML display # Check metadata print(df_raw[['concept', 'balance', 'weight', 'preferred_sign']].head()) # Hierarchy columns — understand parent-child relationships print(df_raw[['label', 'level', 'parent_concept', 'parent_abstract_concept']].head(10))` **See Also:** - Issue #463 - XBRL value transformations and metadata columns - Issue #514 - Parent concept hierarchy columns - [Revenue Segment Hierarchy Guide](https://edgartools.readthedocs.io/en/latest/guides/extract-statements/#understanding-statement-hierarchy) **Returns:** DataFrame with statement data #### get\_concept\_value() `def get_concept_value(self, concept: str) -> Optional[Any]` Get the value for a specific concept. **Parameters:** - `concept`: Concept name to look up **Returns:** Concept value or None **Example:** `statement = xbrl.statements.income_statement() # Render the statement rendered = statement.render() print(rendered) # Convert to DataFrame df = statement.to_dataframe() # Get specific values revenue = statement.get_concept_value("Revenue") net_income = statement.get_concept_value("NetIncomeLoss")` Facts Querying -------------- ### FactsView Provides a view over all XBRL facts with analysis and querying methods. `class FactsView: """View over all facts with analysis methods."""` #### Query Methods #### by\_concept() `def by_concept(self, pattern: str, exact: bool = False) -> FactQuery` Filter facts by concept name. **Parameters:** - `pattern`: Pattern to match against concept names - `exact`: If True, require exact match; otherwise, use regex **Returns:** `FactQuery` object for further filtering #### by\_label() `def by_label(self, pattern: str, exact: bool = False) -> FactQuery` Filter facts by element label. **Parameters:** - `pattern`: Pattern to match against labels - `exact`: If True, require exact match; otherwise, use regex **Returns:** `FactQuery` object for further filtering #### by\_value() `def by_value(self, min_value: float = None, max_value: float = None) -> FactQuery` Filter facts by value range. **Parameters:** - `min_value`: Minimum value threshold - `max_value`: Maximum value threshold **Returns:** `FactQuery` object for further filtering #### by\_period() `def by_period(self, start_date: str = None, end_date: str = None) -> FactQuery` Filter facts by period range. **Parameters:** - `start_date`: Start date (YYYY-MM-DD format) - `end_date`: End date (YYYY-MM-DD format) **Returns:** `FactQuery` object for further filtering #### Analysis Methods #### pivot\_by\_period() `def pivot_by_period(self, concepts: List[str] = None) -> pd.DataFrame` Create a pivot table showing concepts by period. **Parameters:** - `concepts`: List of concepts to include (default: all) **Returns:** DataFrame with concepts as rows and periods as columns #### time\_series() `def time_series(self, concept: str) -> pd.Series` Get time series data for a specific concept. **Parameters:** - `concept`: Concept name **Returns:** pandas Series with time series data #### Data Conversion #### to\_dataframe() `def to_dataframe(self) -> pd.DataFrame` Convert facts to pandas DataFrame. **Returns:** DataFrame with all facts and metadata **Example:** `facts = xbrl.facts # Query by concept revenue_query = facts.by_concept("Revenue") revenue_facts = revenue_query.execute() # Query by label and value large_expenses = facts.by_label("expense").by_value(min_value=1000000) expense_facts = large_expenses.to_dataframe() # Time series analysis revenue_ts = facts.time_series("Revenue") print(revenue_ts.head()) # Pivot analysis pivot_df = facts.pivot_by_period(["Revenue", "NetIncomeLoss"])` ### FactQuery Fluent query builder for filtering and manipulating XBRL facts. `class FactQuery: """A query builder for XBRL facts with fluent interface."""` #### Filtering Methods All filtering methods return `self` for method chaining. #### by\_concept() `def by_concept(self, pattern: str, exact: bool = False) -> FactQuery` #### by\_label() `def by_label(self, pattern: str, exact: bool = False) -> FactQuery` #### by\_value() `def by_value(self, min_value: float = None, max_value: float = None) -> FactQuery` #### by\_period() `def by_period(self, start_date: str = None, end_date: str = None) -> FactQuery` #### by\_statement() `def by_statement(self, statement_type: str) -> FactQuery` Filter facts by statement type. **Parameters:** - `statement_type`: Statement type to filter by **Returns:** `FactQuery` object for method chaining #### Execution Methods #### execute() `def execute(self) -> List[Dict]` Execute the query and return matching facts. **Returns:** List of fact dictionaries #### to\_dataframe() `def to_dataframe(self) -> pd.DataFrame` Execute the query and return results as DataFrame. **Returns:** DataFrame with query results #### first() `def first(self) -> Optional[Dict]` Get the first matching fact. **Returns:** First fact dictionary or None #### count() `def count(self) -> int` Count matching facts without retrieving them. **Returns:** Number of matching facts **Example:** `# Chain multiple filters query = (xbrl.facts .by_concept("Revenue") .by_period(start_date="2023-01-01") .by_value(min_value=1000000)) # Execute in different ways facts_list = query.execute() facts_df = query.to_dataframe() first_fact = query.first() count = query.count()` Multi-Period Analysis --------------------- ### StitchedStatements Interface for accessing multi-period statements that combine data across multiple XBRL documents. `class StitchedStatements: """Interface for multi-period statements."""` #### Statement Access Methods Similar to `Statements` but returns `StitchedStatement` objects. All methods accept these common parameters: * `max_periods` (int): Maximum number of periods to include (default: 8) * `standard` (bool): Whether to use standardized concept labels (default: True) * `use_optimal_periods` (bool): Whether to use entity info for optimal period selection (default: True) * `show_date_range` (bool): Whether to show full date ranges for duration periods (default: False) * `include_dimensions` (bool): Whether to include dimensional segment data (default: False, True for equity/comprehensive income) * `view` (str): Controls dimensional filtering — `"standard"`, `"detailed"`, or `"summary"`. Overrides `include_dimensions` when provided. #### balance\_sheet() `def balance_sheet(self, view=None, **kwargs) -> Optional[StitchedStatement]` #### income\_statement() `def income_statement(self, view=None, **kwargs) -> Optional[StitchedStatement]` #### cashflow\_statement() `def cashflow_statement(self, view=None, **kwargs) -> Optional[StitchedStatement]` #### statement\_of\_equity() `def statement_of_equity(self, view=None, **kwargs) -> Optional[StitchedStatement]` #### comprehensive\_income() `def comprehensive_income(self, view=None, **kwargs) -> Optional[StitchedStatement]` **Example:** `# Multi-period analysis stitched_statements = xbrls.statements income_stmt = stitched_statements.income_statement() # Shows multiple years of data print(income_stmt.render()) # Include dimensional breakdowns (e.g., cost by segment) income_detailed = stitched_statements.income_statement(view="detailed") df = income_detailed.to_dataframe()` ### StitchedStatement Individual statement showing multi-period data with comparative analysis. `class StitchedStatement: """Individual stitched statement showing multi-period data."""` **Constructor Parameters:** - `xbrls`: XBRLS object containing stitched data - `statement_type` (str): Type of statement ('BalanceSheet', 'IncomeStatement', etc.) - `max_periods` (int): Maximum number of periods (default: 8) - `standard` (bool): Use standardized labels (default: True) - `include_dimensions` (bool): Include dimensional data (default: False) - `view` (str): `"standard"`, `"detailed"`, or `"summary"`. Overrides `include_dimensions`. #### Analysis Methods #### render() `def render(self, show_date_range: bool = False) -> Table` Render multi-period statement with rich formatting. #### to\_dataframe() `def to_dataframe(self) -> pd.DataFrame` Convert to DataFrame with periods as columns. Standardization --------------- ### StandardConcept Represents a standardized concept that normalizes company-specific terminology. `class StandardConcept: """Standardized concept representation."""` #### Properties #### name `@property def name(self) -> str` Standardized concept name. #### label `@property def label(self) -> str` Standardized human-readable label. **Example:** `# Standardization is applied automatically in statements statement = xbrl.statements.income_statement() df = statement.to_dataframe() # Check for standardized vs original labels print(df[['label', 'original_label']].head())` Rendering --------- ### RenderedStatement Formatted statement output with rich console display capabilities. `class RenderedStatement: """Rich formatted statement output."""` #### Display Methods #### **str**() `def __str__(self) -> str` Plain text representation of the statement. #### **rich**() `def __rich__(self) -> RichRenderable` Rich console representation with formatting. **Example:** `# Rich rendering in console rendered = xbrl.render_statement("BalanceSheet") print(rendered) # Displays with rich formatting # Plain text for export text_output = str(rendered)` Utility Functions ----------------- ### stitch\_statements() `def stitch_statements(statements: List[Statement]) -> StitchedStatement` Combine multiple statements into a stitched statement. **Parameters:** - `statements`: List of Statement objects to combine **Returns:** `StitchedStatement` object ### render\_stitched\_statement() `def render_stitched_statement(stitched_statement: StitchedStatement, **kwargs) -> RenderedStatement` Render a stitched statement with formatting. **Parameters:** - `stitched_statement`: StitchedStatement to render - `**kwargs`: Rendering options **Returns:** `RenderedStatement` object ### to\_pandas() `def to_pandas(obj: Union[XBRL, Statement, FactsView]) -> pd.DataFrame` Convert various XBRL objects to pandas DataFrame. **Parameters:** - `obj`: Object to convert (XBRL, Statement, or FactsView) **Returns:** DataFrame representation Advanced Usage Examples ----------------------- ### Multi-Period Financial Analysis `from edgar import Company from edgar.xbrl import XBRLS # Get multiple years of data company = Company("AAPL") filings = company.get_filings(form="10-K").head(3) xbrls = XBRLS.from_filings(filings) # Analyze income statement trends income_stmt = xbrls.statements.income_statement() revenue_trend = income_stmt.get_trend("Revenue") revenue_growth = income_stmt.calculate_growth("Revenue") print(f"Revenue Growth: {revenue_growth.iloc[-1]:.2%}")` ### Complex Fact Querying `from edgar import Company from edgar.xbrl import XBRL company = Company("MSFT") filing = company.latest("10-K") xbrl = XBRL.from_filing(filing) # Complex query with multiple filters high_value_revenue = (xbrl.facts .by_concept("Revenue") .by_value(min_value=50000000000) # $50B+ .by_period(start_date="2023-01-01") .to_dataframe()) # Pivot analysis pivot_df = xbrl.facts.pivot_by_period([ "Revenue", "NetIncomeLoss", "OperatingIncomeLoss" ])` ### Statement Comparison `# Compare statements across different companies companies = ["AAPL", "MSFT", "GOOGL"] statements = [] for ticker in companies: company = Company(ticker) filing = company.latest("10-K") xbrl = XBRL.from_filing(filing) if xbrl.statements.income_statement(): statements.append(xbrl.statements.income_statement()) # Create comparison DataFrame comparison_data = [] for stmt in statements: df = stmt.to_dataframe() comparison_data.append(df) # Analyze key metrics across companies key_metrics = ["Revenue", "NetIncomeLoss", "OperatingIncomeLoss"] for metric in key_metrics: print(f"\n{metric} Comparison:") for i, stmt in enumerate(statements): value = stmt.get_concept_value(metric) if value: print(f" {companies[i]}: ${value/1e9:.1f}B")` Import Reference ---------------- `# Core classes from edgar.xbrl import XBRL, XBRLS # Statement classes from edgar.xbrl import Statements, Statement from edgar.xbrl import StitchedStatements, StitchedStatement # Facts querying from edgar.xbrl import FactsView, FactQuery from edgar.xbrl import StitchedFactsView, StitchedFactQuery # Standardization and rendering from edgar.xbrl import StandardConcept, RenderedStatement # Utility functions from edgar.xbrl import stitch_statements, render_stitched_statement, to_pandas` Error Handling -------------- `from edgar.xbrl import XBRL, XBRLFilingWithNoXbrlData try: xbrl = XBRL.from_filing(filing) except XBRLFilingWithNoXbrlData: print("Filing does not contain XBRL data") except Exception as e: print(f"Error parsing XBRL: {e}") # Check for statement availability if xbrl.statements.income_statement(): income_stmt = xbrl.statements.income_statement() df = income_stmt.to_dataframe() else: print("Income statement not found")` XBRL Value Transformations (Issue #463) --------------------------------------- EdgarTools provides a two-layer system for XBRL value handling: ### Value Layers 1. **Raw Values** (default): Values exactly as reported in the XBRL instance document 2. Matches SEC CompanyFacts API 3. Preserves original data for analysis 4. No transformations applied 5. **Presentation Values** (`presentation=True`): Values transformed to match SEC filing HTML display 6. Applies `preferred_sign` transformations from presentation linkbase 7. Cash Flow outflows shown as negative when appropriate 8. Matches how values appear in the official 10-K/10-Q HTML ### Metadata Columns All statement DataFrames include XBRL metadata columns: * **`balance`**: Debit or credit classification from schema (accounting semantics) * **`weight`**: Calculation weight from calculation linkbase (+1.0 or -1.0) * **`preferred_sign`**: Presentation hint from presentation linkbase (+1 or -1) These columns provide transparency about XBRL semantics and enable custom transformations. ### Usage Examples `# Get raw values (default) xbrl = filing.xbrl() statement = xbrl.statements.cash_flow_statement() df_raw = statement.to_dataframe() # PaymentsOfDividends appears as positive (raw XML value) dividends = df_raw[df_raw['concept'].str.contains('PaymentsOfDividends')] print(dividends[['concept', 'balance', 'preferred_sign', '2024-09-30']]) # Output: concept=PaymentsOfDividends, balance=credit, preferred_sign=-1, value=12345000000 (positive) # Get presentation values (matches SEC HTML) df_presentation = statement.to_dataframe(presentation=True) dividends_pres = df_presentation[df_presentation['concept'].str.contains('PaymentsOfDividends')] print(dividends_pres[['concept', '2024-09-30']]) # Output: value=-12345000000 (negative, matches HTML display with parentheses)` ### When to Use Each Mode **Use Raw Values** (default): - Cross-company financial analysis - Data science and machine learning - Comparison with SEC CompanyFacts API - When you need unmodified reported values **Use Presentation Values** (`presentation=True`): - Matching SEC filing HTML display - Creating investor-facing reports - Replicating official financial statement appearance - When users expect "traditional" financial statement signs ### Technical Notes * **Raw values are consistent across companies**: Testing confirmed SEC instance data uses consistent signs * **Metadata always included**: All transformations can be recreated using metadata columns * **No data loss**: Raw values always preserved, transformations are reversible Performance Tips ---------------- 1. **Use specific queries** - Filter facts early to reduce processing time 2. **Cache XBRL objects** - Parsing is expensive, reuse when possible 3. **Limit statement rendering** - Use `max_rows` parameter for large statements 4. **Batch processing** - Use `XBRLS` for efficient multi-period analysis See Also -------- * **[Company API Reference](https://edgartools.readthedocs.io/en/latest/api/company/) ** - Working with company data * **[Filing API Reference](https://edgartools.readthedocs.io/en/latest/api/filing/) ** - Working with individual filings * **[Extract Financial Statements Guide](https://edgartools.readthedocs.io/en/latest/guides/extract-statements/) ** - Practical examples * **[Working with Filing Guide](https://edgartools.readthedocs.io/en/latest/guides/working-with-filing/) ** - Filing workflows Back to top --- # Overview - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/xbrl/#xbrl-financial-data) XBRL Financial Data =================== EdgarTools provides powerful, elegant tools for working with XBRL financial data from SEC filings. Extract financial statements, analyze multi-period trends, and work with complex dimensional data - all with a simple, intuitive API. Quick Start ----------- New to XBRL in EdgarTools? Start here: * **[Extract Financial Statements](https://edgartools.readthedocs.io/en/latest/guides/extract-statements/) ** - Get balance sheets, income statements, and cash flows in 5 minutes (Beginner) * **[Choosing the Right API](https://edgartools.readthedocs.io/en/latest/xbrl/getting-started/choosing-the-right-api/) ** - Understand when to use `filing.xbrl()` vs `company.get_facts()` (Beginner) Common Tasks ------------ Jump to what you need to do: * **[Extract Financial Statements](https://edgartools.readthedocs.io/en/latest/guides/extract-statements/) ** - Get standardized financial statements from any filing * **[Multi-Period Analysis](https://edgartools.readthedocs.io/en/latest/xbrl/guides/multi-period-analysis/) ** - Compare financials across quarters and years * **[Analyze Segments](https://edgartools.readthedocs.io/en/latest/guides/extract-statements/#enhanced-dimensional-display) ** - Work with geographic and business segment breakdowns (Intermediate) * **[Query XBRL Facts](https://edgartools.readthedocs.io/en/latest/api/xbrl/#facts-and-filtering) ** - Search and filter raw XBRL facts programmatically (Advanced) Understanding XBRL ------------------ Conceptual guides explaining how EdgarTools handles XBRL data: * **[Dimension Handling](https://edgartools.readthedocs.io/en/latest/xbrl/concepts/dimension-handling/) ** - How EdgarTools processes segments, scenarios, and other dimensions (Intermediate) * **[Standardization](https://edgartools.readthedocs.io/en/latest/xbrl/concepts/standardization/) ** - How financial statements are normalized across companies (Intermediate) API Reference ------------- Detailed API documentation: * **[XBRL API](https://edgartools.readthedocs.io/en/latest/api/xbrl/) ** - Complete reference for the `XBRL` class and methods * **[EntityFacts API](https://edgartools.readthedocs.io/en/latest/api/entity-facts-reference/) ** - Reference for company-level facts API * **[StatementType Quick Reference](https://edgartools.readthedocs.io/en/latest/StatementType-Quick-Reference/) ** - All available statement types and their uses Getting Help ------------ **Troubleshooting Tips:** * **Statement not found?** Check if the filing contains XBRL data using `filing.xbrl()` * **Unexpected dimensions?** See [Dimension Handling](https://edgartools.readthedocs.io/en/latest/xbrl/concepts/dimension-handling/) for filtering strategies * **Missing values?** Some companies use non-standard tags - use `statement.facts` to explore raw data **Need More Help?** * [Open an issue on GitHub](https://github.com/dgunning/edgartools/issues) - Report bugs or request features * [View Examples](https://edgartools.readthedocs.io/en/latest/guides/) - Browse our collection of practical guides and examples * * * Need help building an XBRL pipeline? The code above extracts XBRL data for one company. Scaling to thousands — with taxonomy normalization, custom extension mapping, and multi-year consistency — is where it gets hard. * **[XBRL consulting for AI & data teams →](https://www.edgar.tools/consulting/xbrl?utm_source=edgartools-docs&utm_medium=see-live&utm_content=xbrl-consulting) ** * **[See all SEC data consulting services →](https://www.edgar.tools/consulting?utm_source=edgartools-docs&utm_medium=see-live&utm_content=xbrl-consulting) ** From the creator of edgartools. [Book a call →](https://www.edgar.tools/consulting/xbrl?utm_source=edgartools-docs&utm_medium=see-live&utm_content=xbrl-consulting#contact) **Note:** EdgarTools handles the complexity of XBRL so you don't have to. If you're new to XBRL, don't worry - our guides assume no prior knowledge of XBRL or SEC filings. Back to top --- # Institutional Holdings (13F) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/thirteenf-data-object-guide/#13f-holdings-parse-sec-institutional-portfolio-filings-with-python) 13F Holdings: Parse SEC Institutional Portfolio Filings with Python =================================================================== See what the big funds are buying. SEC 13F filings disclose the equity holdings of institutional managers with over $100M in assets -- every quarter, publicly available. EdgarTools parses these filings into structured Python objects so you can analyze portfolios in a few lines of code. `from edgar import get_filings filings = get_filings(form="13F-HR") report = filings[0].obj() report` ![13F holdings report parsed with Python edgartools](https://edgartools.readthedocs.io/en/latest/images/thirteenf.webp) Three lines to get a fully parsed holdings report with management company, total portfolio value, and every position. * * * Access Holdings Data -------------------- The `.holdings` property returns a DataFrame with one row per security, aggregated across managers, sorted by value: `report.holdings` | Column | What it is | | --- | --- | | `Issuer` | Company name (`"APPLE INC"`) | | `Ticker` | Resolved ticker symbol (`"AAPL"`) | | `Value` | Market value in **thousands** of dollars | | `SharesPrnAmount` | Share count or principal amount | | `Cusip` | 9-character CUSIP | | `Type` | `"Shares"` or `"Principal"` | | `PutCall` | `"PUT"`, `"CALL"`, or empty | Values are in thousands -- the SEC's reporting unit. `Value` of 135,364 means $135.4 million. * * * Compare 13F Holdings Quarter-over-Quarter ----------------------------------------- One call to see what changed: `report.compare_holdings()` ![Python 13F holdings quarter-over-quarter comparison](https://edgartools.readthedocs.io/en/latest/images/thirteenf_compare.webp) Every position gets a status: **NEW**, **CLOSED**, **INCREASED**, **DECREASED**, or **UNCHANGED**. Results are sorted by absolute value change so the biggest moves appear first. `comparison = report.compare_holdings() # Dig into the data df = comparison.data new_buys = df[df['Status'] == 'NEW'] exits = df[df['Status'] == 'CLOSED']` The comparison DataFrame includes `Shares`, `PrevShares`, `ShareChange`, `ShareChangePct`, `Value`, `PrevValue`, `ValueChange`, `ValueChangePct`, and `Status`. * * * Track Holdings Trends Across Quarters ------------------------------------- See how positions evolve across quarters with sparkline visualizations: `report.holding_history(periods=4)` ![13F holdings history with sparkline trends in Python](https://edgartools.readthedocs.io/en/latest/images/thirteenf_history.webp) Each row shows share counts per quarter and a Unicode sparkline (`▁▂▃▅▇`) so you can spot trends at a glance. `history = report.holding_history(periods=4) df = history.data # Full DataFrame with one column per quarter` * * * Using View Objects in Your Own App ---------------------------------- `holdings_view()`, `compare_holdings()`, and `holding_history()` all return view objects that render in the terminal via Rich but also support iteration, indexing, and access to the underlying DataFrame. This makes them useful for building your own dashboards, reports, or exports. All three views share the same interface: `view = report.holdings_view() comparison = report.compare_holdings() history = report.holding_history(periods=4) # Iterate rows as dicts for row in view: print(row['Ticker'], row['Value']) # Index a single row (returns dict) view[0] # Slice (returns DataFrame) view[:10] # Length len(view) # Access the full DataFrame directly view.data comparison.data history.data` Each view also carries metadata useful for rendering headers: | View | Metadata | | --- | --- | | `HoldingsView` | `.display_limit` | | `HoldingsComparison` | `.current_period`, `.previous_period`, `.manager_name` | | `HoldingsHistory` | `.periods` (list of quarter dates), `.manager_name` | * * * Look Up a Specific Fund ----------------------- `from edgar import Company berkshire = Company("BRK.A") filing = berkshire.get_filings(form="13F-HR").latest(1) report = filing.obj() print(report.management_company_name) # "Berkshire Hathaway Inc" print(f"${report.total_value:,}K across {report.total_holdings} holdings")` * * * Common Analysis Patterns ------------------------ ### Portfolio concentration `h = report.holdings total = h['Value'].sum() h['Weight'] = (h['Value'] / total * 100).round(2) h[['Ticker', 'Issuer', 'Value', 'Weight']].head(10)` ### Options positions `report.holdings.query("PutCall in ['PUT', 'CALL']")` ### Previous quarter's full report `previous = report.previous_holding_report() # Returns a ThirteenF or None previous.holdings` * * * Multi-Manager Filings --------------------- Large institutions (Bank of America, State Street) file consolidated 13F reports. The `holdings` property automatically aggregates across all managers. If you need per-manager detail, use `infotable` instead: `report.infotable # Disaggregated: one row per manager-security pair report.holdings # Aggregated: one row per security (recommended) # Example: Berkshire Hathaway # infotable: ~121 rows (3 managers x ~40 securities) # holdings: ~40 rows (aggregated by CUSIP) # See who the other managers are for mgr in report.other_managers: print(f"{mgr.name} (CIK: {mgr.cik})")` * * * Metadata Quick Reference ------------------------ | Property | Returns | Example | | --- | --- | --- | | `management_company_name` | Company that filed | `"Berkshire Hathaway Inc"` | | `report_period` | Quarter end date | `"2024-03-31"` | | `filing_date` | Date filed | `"2024-05-15"` | | `total_value` | Portfolio value ($000s) | `Decimal('313218000')` | | `total_holdings` | Number of positions | `40` | | `filing_signer_name` | Who signed | `"Marc D. Hamburg"` | | `filing_signer_title` | Signer's title | `"Senior Vice President"` | | `form` | Form type | `"13F-HR"` | | `accession_number` | SEC accession no. | `"0000950123-24-007092"` | | `has_infotable()` | Has holdings data? | `True` for 13F-HR, `False` for 13F-NT | * * * Methods Quick Reference ----------------------- | Call | Returns | What it does | | --- | --- | --- | | `report.holdings` | `DataFrame` | Aggregated holdings, one row per security | | `report.infotable` | `DataFrame` | Raw holdings, disaggregated by manager | | `report.holdings_view()` | `HoldingsView` | Rich-renderable, iterable holdings | | `report.compare_holdings()` | `HoldingsComparison` | Quarter-over-quarter changes with status labels | | `report.holding_history(periods=4)` | `HoldingsHistory` | Multi-quarter share trends with sparklines | | `report.previous_holding_report()` | `ThirteenF` | Previous quarter's 13F object | | `report.other_managers` | `list[OtherManager]` | Affiliated managers in consolidated filings | | `report.get_portfolio_managers()` | `list[dict]` | Curated lookup of known portfolio managers | * * * Things to Know -------------- **Values are in thousands.** The SEC requires 13F values in $000s. A `Value` of 135,364 is $135.4 million. **`holdings` vs `infotable`.** Use `holdings` (aggregated by CUSIP) for portfolio analysis. Use `infotable` only when you need per-manager detail in multi-manager filings. **Ticker resolution.** Tickers are resolved from CUSIPs. Most resolve correctly, but delisted or obscure securities may show as blank. **Pre-2013 filings use TXT format.** EdgarTools parses both XML (2013+) and TXT (2012 and earlier) transparently, but older filings may have fewer columns. **13F-NT means no holdings.** Notice filings indicate the manager had nothing to report. `has_infotable()` returns `False`. **Report period vs filing date.** The `report_period` is the quarter end. The `filing_date` can be up to 45 days later. Some managers file multiple historical periods on the same day. * * * Related ------- * [Institutional Holdings Guide](https://edgartools.readthedocs.io/en/latest/13f-filings/) -- workflow-oriented guide for finding, analyzing, and comparing 13F holdings * [Working with Filings](https://edgartools.readthedocs.io/en/latest/guides/working-with-filing/) -- general filing access patterns Back to top --- # Overview - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/xbrl/#xbrl-financial-data) XBRL Financial Data =================== EdgarTools provides powerful, elegant tools for working with XBRL financial data from SEC filings. Extract financial statements, analyze multi-period trends, and work with complex dimensional data - all with a simple, intuitive API. Quick Start ----------- New to XBRL in EdgarTools? Start here: * **[Extract Financial Statements](https://edgartools.readthedocs.io/en/stable/guides/extract-statements/) ** - Get balance sheets, income statements, and cash flows in 5 minutes (Beginner) * **[Choosing the Right API](https://edgartools.readthedocs.io/en/stable/xbrl/getting-started/choosing-the-right-api/) ** - Understand when to use `filing.xbrl()` vs `company.get_facts()` (Beginner) Common Tasks ------------ Jump to what you need to do: * **[Extract Financial Statements](https://edgartools.readthedocs.io/en/stable/guides/extract-statements/) ** - Get standardized financial statements from any filing * **[Multi-Period Analysis](https://edgartools.readthedocs.io/en/stable/xbrl/guides/multi-period-analysis/) ** - Compare financials across quarters and years * **[Analyze Segments](https://edgartools.readthedocs.io/en/stable/guides/extract-statements/#enhanced-dimensional-display) ** - Work with geographic and business segment breakdowns (Intermediate) * **[Query XBRL Facts](https://edgartools.readthedocs.io/en/stable/api/xbrl/#facts-and-filtering) ** - Search and filter raw XBRL facts programmatically (Advanced) Understanding XBRL ------------------ Conceptual guides explaining how EdgarTools handles XBRL data: * **[Dimension Handling](https://edgartools.readthedocs.io/en/stable/xbrl/concepts/dimension-handling/) ** - How EdgarTools processes segments, scenarios, and other dimensions (Intermediate) * **[Standardization](https://edgartools.readthedocs.io/en/stable/xbrl/concepts/standardization/) ** - How financial statements are normalized across companies (Intermediate) API Reference ------------- Detailed API documentation: * **[XBRL API](https://edgartools.readthedocs.io/en/stable/api/xbrl/) ** - Complete reference for the `XBRL` class and methods * **[EntityFacts API](https://edgartools.readthedocs.io/en/stable/api/entity-facts-reference/) ** - Reference for company-level facts API * **[StatementType Quick Reference](https://edgartools.readthedocs.io/en/stable/StatementType-Quick-Reference/) ** - All available statement types and their uses Getting Help ------------ **Troubleshooting Tips:** * **Statement not found?** Check if the filing contains XBRL data using `filing.xbrl()` * **Unexpected dimensions?** See [Dimension Handling](https://edgartools.readthedocs.io/en/stable/xbrl/concepts/dimension-handling/) for filtering strategies * **Missing values?** Some companies use non-standard tags - use `statement.facts` to explore raw data **Need More Help?** * [Open an issue on GitHub](https://github.com/dgunning/edgartools/issues) - Report bugs or request features * [View Examples](https://edgartools.readthedocs.io/en/stable/guides/) - Browse our collection of practical guides and examples * * * Need help building an XBRL pipeline? The code above extracts XBRL data for one company. Scaling to thousands — with taxonomy normalization, custom extension mapping, and multi-year consistency — is where it gets hard. * **[XBRL consulting for AI & data teams →](https://www.edgar.tools/consulting/xbrl?utm_source=edgartools-docs&utm_medium=see-live&utm_content=xbrl-consulting) ** * **[See all SEC data consulting services →](https://www.edgar.tools/consulting?utm_source=edgartools-docs&utm_medium=see-live&utm_content=xbrl-consulting) ** From the creator of edgartools. [Book a call →](https://www.edgar.tools/consulting/xbrl?utm_source=edgartools-docs&utm_medium=see-live&utm_content=xbrl-consulting#contact) **Note:** EdgarTools handles the complexity of XBRL so you don't have to. If you're new to XBRL, don't worry - our guides assume no prior knowledge of XBRL or SEC filings. Back to top --- # Current Filings - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/current-filings/#get-todays-sec-filings-real-time-edgar-filing-access) Get Today's SEC Filings: Real-Time EDGAR Filing Access ====================================================== Overview -------- Current filings represent the most recently submitted documents to the SEC, updated in real-time as companies file their reports. This guide shows you how to access, filter, and efficiently process current filings using edgartools. Quick Start ----------- ### Basic Usage `from edgar import get_current_filings # Get the most recent filings (default: 100 filings) current = get_current_filings() print(f"Found {len(current)} recent filings") # Display the first few filings for filing in current[:5]: print(f"{filing.form}: {filing.company} - {filing.filing_date}")` **Output:** `Found 100 recent filings 8-K: Apple Inc. - 2025-01-14 10-Q: Microsoft Corporation - 2025-01-14 4: BEZOS JEFFREY P - 2025-01-14 13F-HR: Berkshire Hathaway Inc - 2025-01-14 S-3: Tesla, Inc. - 2025-01-14` ### Filter by Form Type `# Get only Form 8-K current events current_8k = get_current_filings(form='8-K') # Get only insider trading forms (Forms 3, 4, 5) current_insider = get_current_filings(form='4') # Get quarterly and annual reports current_reports = get_current_filings(form='10-K')` Understanding Current Filings ----------------------------- ### What Are Current Filings? Current filings are the most recently submitted documents to the SEC, typically updated every few minutes during business hours. They include: * **Form 8-K**: Current events and corporate changes * **Forms 3, 4, 5**: Insider trading transactions * **10-K/10-Q**: Annual and quarterly reports * **13F**: Institutional investment manager holdings * **S-1, S-3**: Registration statements * **And many more...** ### Pagination System Current filings are delivered in pages to manage large volumes: `# Default: Get first 100 filings current = get_current_filings(page_size=100) # Get more filings per page (up to 100) current = get_current_filings(page_size=80) # Navigate to next page next_page = current.next() if next_page: print(f"Next page has {len(next_page)} filings")` Core Functions -------------- ### `get_current_filings()` Get a single page of current filings with filtering options. `def get_current_filings(form: str = '', owner: str = 'include', page_size: int = 100) -> CurrentFilings:` **Parameters:** - `form` (str): Filter by form type (e.g., "8-K", "10-K", "4") - `owner` (str): Owner filter - "include", "exclude", or "only" - `page_size` (int): Filings per page (10, 20, 40, 80, or 100) **Returns:** `CurrentFilings` object with pagination capabilities ### `iter_current_filings_pages()` Iterator that yields pages of current filings until exhausted. `from edgar import iter_current_filings_pages # Process all current 8-K filings page by page for page in iter_current_filings_pages(form="8-K"): print(f"Processing {len(page)} 8-K filings") for filing in page: # Process each filing print(f" {filing.company}: {filing.filing_date}") # Break after first few pages for demo if page.current_page >= 3: break` ### `get_all_current_filings()` Get ALL current filings by automatically iterating through all pages. `from edgar import get_all_current_filings # Get all current Form 4 filings (may be thousands) all_form4 = get_all_current_filings(form="4") print(f"Total Form 4 filings: {len(all_form4)}") # Get all current filings (no form filter) all_current = get_all_current_filings() print(f"Total current filings: {len(all_current)}")` **⚠️ Performance Note:** This function downloads ALL available current filings, which can be thousands of documents. Use with appropriate filters. Filtering Options ----------------- ### By Form Type `# Specific form types form_8k = get_current_filings(form="8-K") form_10k = get_current_filings(form="10-K") form_4 = get_current_filings(form="4") # Form families work too quarterly_reports = get_current_filings(form="10-Q")` ### By Owner Type Control whether to include filings from investment managers: `# Include all filings (default) all_filings = get_current_filings(owner="include") # Exclude ownership filings (e.g., Form 4, 144) public_only = get_current_filings(owner="exclude") # Only ownership filings (e.g., Form 4, 144) managers_only = get_current_filings(owner="only")` ### By Page Size Choose how many filings to get per request: `# Small batches for quick processing small_batch = get_current_filings(page_size=20) # Large batches for efficiency large_batch = get_current_filings(page_size=100) # Maximum` Real-World Examples ------------------- ### Example 1: Monitor Recent 8-K Events `from edgar import get_all_current_filings from datetime import datetime def monitor_current_events(): """Monitor recent 8-K filings for significant events.""" # Get recent 8-K filings current_8k = get_all_current_filings(form="8-K") print(f"📈 Monitoring {len(current_8k)} recent 8-K filings") print(f"Last updated: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}") print("-" * 60) for filing in current_8k: # Show key information print(f"{filing.company}") print(f" Form: {filing.form}") print(f" Filed: {filing.filing_date}") print(f" URL: {filing.document_url}") print() monitor_current_events()` ### Example 2: Track Insider Trading Activity `from edgar import get_all_current_filings import pandas as pd def analyze_insider_activity(): """Analyze current insider trading patterns.""" # Get all current Form 4 filings print("📊 Downloading all current Form 4 filings...") insider_filings = get_all_current_filings(form="4") print(f"Found {len(insider_filings)} insider trading filings") # Convert to DataFrame for analysis df = insider_filings.to_pandas() # Analyze by company company_counts = df['company'].value_counts().head(10) print("\n🏢 Top 10 Companies by Filing Volume:") for company, count in company_counts.items(): print(f" {company}: {count} filings") # Analyze by filing date daily_counts = df['filing_date'].value_counts().sort_index() print(f"\n📅 Daily Filing Counts (last {len(daily_counts)} days):") for date, count in daily_counts.tail(7).items(): print(f" {date}: {count} filings") return df # Run the analysis insider_df = analyze_insider_activity()` ### Example 3: Real-Time Filing Feed `from edgar import get_current_filings import time def real_time_filing_feed(max_iterations=10): """Create a real-time feed of new filings.""" seen_filings = set() iteration = 0 print("🔄 Starting real-time filing feed...") print("Press Ctrl+C to stop\n") try: while iteration < max_iterations: # Get latest filings current = get_current_filings(page_size=20) new_filings = [] for filing in current: filing_id = filing.accession_no if filing_id not in seen_filings: new_filings.append(filing) seen_filings.add(filing_id) if new_filings: print(f"🆕 {len(new_filings)} new filings detected:") for filing in new_filings: print(f" {filing.form}: {filing.company}") print() else: print("⏳ No new filings found, waiting...") # Wait before next check time.sleep(30) # Check every 30 seconds iteration += 1 except KeyboardInterrupt: print("\n✋ Feed stopped by user") # Run the feed (limited iterations for demo) real_time_filing_feed()` Performance Considerations -------------------------- ### Memory Usage `# Memory efficient: Process page by page total_processed = 0 for page in iter_current_filings_pages(form="8-K"): # Process this page total_processed += len(page) # Page goes out of scope, memory is freed print(f"Processed {total_processed} total filings") # Memory intensive: Load all at once all_filings = get_all_current_filings() # May use significant memory` ### Network Efficiency `# Efficient: Larger page sizes reduce requests efficient = get_current_filings(page_size=100) # 1 request # Less efficient: Smaller pages mean more requests less_efficient = get_current_filings(page_size=10) # May need 10 requests for same data` ### Rate Limiting The SEC imposes rate limits, so avoid rapid consecutive requests: `import time # Good: Natural pacing between requests for page in iter_current_filings_pages(): # Process page time.sleep(0.1) # Brief pause between pages # Bad: Rapid fire requests (may hit rate limits) for i in range(100): page = get_current_filings() # Don't do this!` Choosing the Right Function --------------------------- ### Use `get_current_filings()` when: * ✅ You want a quick sample of recent filings * ✅ Building pagination in your own interface * ✅ Memory usage is a concern * ✅ You only need the first page or two ### Use `iter_current_filings_pages()` when: * ✅ You want to process all filings but control memory usage * ✅ You need page-by-page processing logic * ✅ You want to limit total pages processed * ✅ Building streaming or incremental processing ### Use `get_all_current_filings()` when: * ✅ You need the complete dataset for analysis * ✅ Memory usage is not a constraint * ✅ You want to convert to pandas DataFrame * ✅ Building bulk analysis or reporting Error Handling -------------- ### Common Issues and Solutions `from edgar import get_current_filings import time def robust_current_filings(form="", max_retries=3): """Get current filings with error handling.""" for attempt in range(max_retries): try: return get_current_filings(form=form) except ConnectionError as e: print(f"⚠️ Connection error (attempt {attempt + 1}): {e}") if attempt < max_retries - 1: time.sleep(2 ** attempt) # Exponential backoff else: raise except Exception as e: print(f"❌ Unexpected error: {e}") raise # Usage try: filings = robust_current_filings(form="8-K") print(f"✅ Successfully retrieved {len(filings)} filings") except Exception as e: print(f"💥 Failed to get filings: {e}")` Best Practices -------------- ### 1\. Use Appropriate Filters `# Good: Specific filtering reduces data and improves performance insider_filings = get_current_filings(form="4") corporate_events = get_current_filings(form="8-K") # Okay: General purpose but processes more data all_filings = get_current_filings()` ### 2\. Handle Pagination Properly `# Good: Check for None before processing next page current_page = get_current_filings() while current_page is not None: # Process current page for filing in current_page: print(f"Processing {filing.company}") # Get next page current_page = current_page.next() # Bad: Assuming next() always returns data # This could cause infinite loops or errors` ### 3\. Be Respectful of SEC Resources `# Good: Process in reasonable batches with pauses for page in iter_current_filings_pages(page_size=100): # Process page time.sleep(0.1) # Brief pause # Good: Cache results when possible cached_filings = get_all_current_filings(form="8-K") # Reuse cached_filings instead of re-downloading` Common Use Cases ---------------- ### Research and Analysis * **Market surveillance**: Monitor 8-K filings for material events * **Insider tracking**: Analyze Form 4 patterns for trading insights * **Compliance monitoring**: Track filing compliance across companies ### Application Development * **Filing alerts**: Build notifications for specific form types * **Data pipelines**: Integrate current filings into larger workflows * **Dashboard feeds**: Power real-time filing displays ### Academic Research * **Event studies**: Analyze market reactions to filing events * **Disclosure analysis**: Study timing and content patterns * **Regulatory compliance**: Research filing behavior patterns Summary ------- Current filings provide real-time access to the latest SEC documents, enabling immediate analysis of corporate events, insider trading, and regulatory submissions. The three main functions offer flexibility for different use cases: * **`get_current_filings()`**: Single page access with pagination control * **`iter_current_filings_pages()`**: Memory-efficient iteration through all pages * **`get_all_current_filings()`**: Bulk access to complete current filing dataset Choose the approach that best fits your memory constraints, processing requirements, and analysis goals. Next Steps ---------- * **Guide**: [Working with Filings](https://edgartools.readthedocs.io/en/stable/guides/current-filings/working-with-filings.md) * **API Reference**: [Filings API](https://edgartools.readthedocs.io/en/stable/api/filings/) Back to top --- # Cloud Storage (S3) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/cloud-storage/#cloud-storage-integration-guide) Cloud Storage Integration Guide =============================== This guide covers integrating EdgarTools with cloud storage providers (AWS S3, Google Cloud Storage, Azure Blob Storage) and S3-compatible services (Cloudflare R2, MinIO, DigitalOcean Spaces). Why Cloud Storage? ------------------ Cloud storage provides several advantages over local storage: | Benefit | Description | | --- | --- | | **Scalability** | Store terabytes of SEC data without local disk constraints | | **Team Sharing** | Multiple users/services access the same dataset | | **Durability** | Cloud providers offer 99.999999999% durability | | **Cost Efficiency** | Pay only for storage used; cheaper than provisioning servers | | **Global Access** | Access data from anywhere, any environment | Integration Approaches ---------------------- EdgarTools supports cloud storage through three mechanisms: 1. **`use_cloud_storage()`** - Native cloud integration via fsspec for **reading and writing** (recommended) 2. **`EDGAR_DATA_URL`** - Point to any HTTP endpoint for **reading** data 3. **`EDGAR_LOCAL_DATA_DIR` + FUSE** - Mount cloud storage as a local path (legacy) ### Approach Comparison | Feature | Native (`use_cloud_storage`) | EDGAR\_DATA\_URL | FUSE Mount | | --- | --- | --- | --- | | **Setup Complexity** | Simple | Simple | Complex | | **Read Data** | Yes | Yes | Yes | | **Write Data** | Yes | No | Yes | | **Requires Mount** | No | No | Yes | | **Platform Support** | All | All | Linux/macOS | | **Best For** | Full cloud integration | Read-only HTTP | Legacy systems | * * * Approach 1: EDGAR\_DATA\_URL (Read-Only) ---------------------------------------- The simplest approach for read-only access. Point EdgarTools to an HTTP endpoint serving your SEC data. ### How It Works `import os os.environ['EDGAR_DATA_URL'] = 'https://your-bucket.s3.amazonaws.com/edgar-data/' os.environ['EDGAR_USE_LOCAL_DATA'] = '1' from edgar import Company company = Company("AAPL") # Fetches from your S3 bucket` ### Setting Up S3 Static Website Hosting #### Step 1: Create and Configure S3 Bucket `# Create bucket aws s3 mb s3://my-edgar-data --region us-east-1 # Enable static website hosting aws s3 website s3://my-edgar-data \ --index-document index.html \ --error-document error.html` #### Step 2: Set Bucket Policy for Public Read Create `bucket-policy.json`: `{ "Version": "2012-10-17", "Statement": [ { "Sid": "PublicReadGetObject", "Effect": "Allow", "Principal": "*", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::my-edgar-data/*" } ] }` Apply the policy: `aws s3api put-bucket-policy \ --bucket my-edgar-data \ --policy file://bucket-policy.json` #### Step 3: Upload Your Data `# Sync local edgar data to S3 aws s3 sync ~/.edgar s3://my-edgar-data/ --storage-class STANDARD_IA` #### Step 4: Configure EdgarTools `import os # S3 static website URL format os.environ['EDGAR_DATA_URL'] = 'http://my-edgar-data.s3-website-us-east-1.amazonaws.com/' os.environ['EDGAR_USE_LOCAL_DATA'] = '1' from edgar import Company company = Company("AAPL") # Now reads from S3` ### Google Cloud Storage Setup `# Create bucket with uniform access gsutil mb -l us-central1 gs://my-edgar-data # Make bucket publicly readable gsutil iam ch allUsers:objectViewer gs://my-edgar-data # Upload data gsutil -m rsync -r ~/.edgar gs://my-edgar-data/` Configure EdgarTools: `import os os.environ['EDGAR_DATA_URL'] = 'https://storage.googleapis.com/my-edgar-data/' os.environ['EDGAR_USE_LOCAL_DATA'] = '1'` ### Azure Blob Storage Setup `# Create storage account and container az storage account create --name myedgardata --resource-group mygroup az storage container create --name edgar --account-name myedgardata --public-access blob # Upload data az storage blob upload-batch \ --account-name myedgardata \ --destination edgar \ --source ~/.edgar` Configure EdgarTools: `import os os.environ['EDGAR_DATA_URL'] = 'https://myedgardata.blob.core.windows.net/edgar/' os.environ['EDGAR_USE_LOCAL_DATA'] = '1'` ### Adding CloudFront CDN (Recommended for Production) For better performance and reduced S3 costs, add CloudFront: `# Create CloudFront distribution pointing to S3 aws cloudfront create-distribution \ --origin-domain-name my-edgar-data.s3.amazonaws.com \ --default-root-object index.html` Then use your CloudFront URL: `os.environ['EDGAR_DATA_URL'] = 'https://d1234567890.cloudfront.net/'` * * * Approach 2: FUSE Mount (Read/Write) ----------------------------------- For full read/write access, mount cloud storage as a local filesystem using FUSE (Filesystem in Userspace). ### FUSE Tool Comparison | Tool | Provider | Performance | Caching | Notes | | --- | --- | --- | --- | --- | | **s3fs-fuse** | AWS S3 | Moderate | Basic | Most compatible | | **goofys** | AWS S3 | Fast | Aggressive | Performance-focused | | **rclone mount** | All providers | Good | Configurable | Most versatile | | **gcsfuse** | Google Cloud | Good | Metadata | Official GCS tool | | **blobfuse2** | Azure | Good | File cache | Official Azure tool | ### s3fs-fuse Setup (AWS S3) #### Installation `# Ubuntu/Debian sudo apt-get install s3fs # macOS brew install s3fs # From source git clone https://github.com/s3fs-fuse/s3fs-fuse.git cd s3fs-fuse && ./autogen.sh && ./configure && make && sudo make install` #### Configuration Create credentials file: `echo "ACCESS_KEY_ID:SECRET_ACCESS_KEY" > ~/.passwd-s3fs chmod 600 ~/.passwd-s3fs` #### Mount the Bucket `# Create mount point mkdir -p /mnt/edgar-data # Mount with caching for better performance s3fs my-edgar-bucket /mnt/edgar-data \ -o passwd_file=~/.passwd-s3fs \ -o url=https://s3.amazonaws.com \ -o use_cache=/tmp/s3fs-cache \ -o ensure_diskfree=1024 \ -o parallel_count=15` #### Configure EdgarTools `import os os.environ['EDGAR_LOCAL_DATA_DIR'] = '/mnt/edgar-data' os.environ['EDGAR_USE_LOCAL_DATA'] = '1' from edgar import download_filings download_filings("2025-01-15") # Writes directly to S3!` ### goofys Setup (High Performance S3) goofys offers better performance than s3fs at the cost of some POSIX compliance. #### Installation `# Download binary wget https://github.com/kahing/goofys/releases/latest/download/goofys chmod +x goofys sudo mv goofys /usr/local/bin/` #### Mount `# Uses standard AWS credentials (~/.aws/credentials) goofys my-edgar-bucket /mnt/edgar-data # With specific profile goofys --profile production my-edgar-bucket /mnt/edgar-data # With caching goofys --stat-cache-ttl 1h --type-cache-ttl 1h my-edgar-bucket /mnt/edgar-data` ### rclone mount (Multi-Provider) rclone supports 40+ cloud storage providers with a unified interface. #### Installation `# Linux curl https://rclone.org/install.sh | sudo bash # macOS brew install rclone` #### Configure Provider `# Interactive configuration rclone config # Example: Configure S3 # Name: edgar-s3 # Type: s3 # Provider: AWS # Access key: (your key) # Secret key: (your secret) # Region: us-east-1` #### Mount `# Basic mount rclone mount edgar-s3:my-edgar-bucket /mnt/edgar-data # With VFS caching (recommended) rclone mount edgar-s3:my-edgar-bucket /mnt/edgar-data \ --vfs-cache-mode full \ --vfs-cache-max-age 24h \ --vfs-read-ahead 128M \ --buffer-size 128M \ --daemon` ### gcsfuse Setup (Google Cloud) `# Installation export GCSFUSE_REPO=gcsfuse-$(lsb_release -c -s) echo "deb https://packages.cloud.google.com/apt $GCSFUSE_REPO main" | sudo tee /etc/apt/sources.list.d/gcsfuse.list curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - sudo apt-get update && sudo apt-get install gcsfuse # Mount gcsfuse --implicit-dirs my-edgar-bucket /mnt/edgar-data` ### blobfuse2 Setup (Azure) `# Installation wget https://packages.microsoft.com/config/ubuntu/22.04/packages-microsoft-prod.deb sudo dpkg -i packages-microsoft-prod.deb sudo apt-get update && sudo apt-get install blobfuse2 # Create config file cat > ~/blobfuse2.yaml << EOF allow-other: true logging: type: syslog level: log_warning components: - libfuse - file_cache - attr_cache - azstorage libfuse: attribute-expiration-sec: 120 entry-expiration-sec: 120 file_cache: path: /tmp/blobfuse2 timeout-sec: 120 max-size-mb: 4096 azstorage: type: block account-name: myedgardata account-key: YOUR_ACCOUNT_KEY container: edgar EOF # Mount blobfuse2 mount /mnt/edgar-data --config-file=~/blobfuse2.yaml` ### Systemd Service (Auto-Mount on Boot) Create `/etc/systemd/system/edgar-s3.service`: `[Unit] Description=Mount S3 Edgar Data After=network-online.target [Service] Type=forking User=edgar ExecStart=/usr/local/bin/goofys -o allow_other my-edgar-bucket /mnt/edgar-data ExecStop=/bin/fusermount -u /mnt/edgar-data Restart=on-failure [Install] WantedBy=multi-user.target` Enable: `sudo systemctl enable edgar-s3 sudo systemctl start edgar-s3` * * * S3-Compatible Services ---------------------- ### Cloudflare R2 R2 offers S3-compatible storage with zero egress fees. #### Key Configuration R2 requires `region_name='auto'`: `# s3fs with R2 echo "R2_ACCESS_KEY:R2_SECRET_KEY" > ~/.passwd-r2 s3fs my-bucket /mnt/edgar-data \ -o passwd_file=~/.passwd-r2 \ -o url=https://ACCOUNT_ID.r2.cloudflarestorage.com \ -o use_path_request_style` #### rclone Configuration for R2 `rclone config # Name: edgar-r2 # Type: s3 # Provider: Cloudflare # access_key_id: (R2 access key) # secret_access_key: (R2 secret key) # endpoint: https://ACCOUNT_ID.r2.cloudflarestorage.com # acl: private` Mount: `rclone mount edgar-r2:my-edgar-bucket /mnt/edgar-data \ --vfs-cache-mode full` #### EDGAR\_DATA\_URL with R2 For read-only access via R2's public URL: `import os # Enable public access on your R2 bucket first os.environ['EDGAR_DATA_URL'] = 'https://pub-xxxxx.r2.dev/' os.environ['EDGAR_USE_LOCAL_DATA'] = '1'` ### MinIO MinIO is perfect for on-premises or private cloud deployments. `# s3fs with MinIO s3fs my-bucket /mnt/edgar-data \ -o passwd_file=~/.passwd-minio \ -o url=https://minio.example.com \ -o use_path_request_style # rclone config # Provider: Minio # Endpoint: https://minio.example.com` ### DigitalOcean Spaces `# rclone config # Provider: DigitalOcean # Endpoint: nyc3.digitaloceanspaces.com` * * * Hybrid Architecture Pattern --------------------------- Combine the best of both approaches for optimal performance: `┌─────────────────────────────────────────────────────────────┐ │ Hybrid Architecture │ ├─────────────────────────────────────────────────────────────┤ │ │ │ WRITES (Download/Sync) READS (Analysis) │ │ ┌─────────────────┐ ┌─────────────────┐ │ │ │ FUSE Mount │ │ EDGAR_DATA_URL │ │ │ │ (s3fs/rclone) │ │ + CloudFront │ │ │ └────────┬────────┘ └────────┬────────┘ │ │ │ │ │ │ ▼ ▼ │ │ ┌──────────────────────────────────────────────────┐ │ │ │ S3 Bucket (Origin) │ │ │ │ my-edgar-data │ │ │ └──────────────────────────────────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────┘` ### Implementation **Download Server (writes to S3):** `# Mount S3 for writing goofys my-edgar-bucket /mnt/edgar-data # Configure EdgarTools export EDGAR_LOCAL_DATA_DIR=/mnt/edgar-data export EDGAR_USE_LOCAL_DATA=1` `from edgar import download_filings download_filings("2025-01-01:2025-01-31") # Writes to S3` **Analysis Clients (reads via HTTP):** `import os # Fast reads via CloudFront os.environ['EDGAR_DATA_URL'] = 'https://d1234567890.cloudfront.net/' os.environ['EDGAR_USE_LOCAL_DATA'] = '1' from edgar import Company, get_filings # All reads go through CloudFront CDN filings = get_filings(form="10-K", year=2024)` * * * Sync Strategies --------------- ### Initial Bulk Upload `# Parallel upload with rclone rclone copy ~/.edgar edgar-s3:my-edgar-bucket \ --transfers 32 \ --checkers 16 \ --progress # Or with AWS CLI aws s3 sync ~/.edgar s3://my-edgar-bucket \ --storage-class STANDARD_IA` ### Incremental Daily Sync Create a cron job for daily updates: `# /etc/cron.d/edgar-sync 0 6 * * * edgar /usr/local/bin/edgar-daily-sync.sh` `edgar-daily-sync.sh`: `#!/bin/bash set -e # Download yesterday's filings locally first export EDGAR_LOCAL_DATA_DIR=/tmp/edgar-staging python -c " from edgar import download_filings from datetime import datetime, timedelta yesterday = (datetime.now() - timedelta(days=1)).strftime('%Y-%m-%d') download_filings(yesterday) " # Sync to S3 rclone sync /tmp/edgar-staging/filings edgar-s3:my-edgar-bucket/filings \ --transfers 16 \ --progress # Cleanup rm -rf /tmp/edgar-staging` ### Bidirectional Sync For teams with multiple download nodes: `# Use rclone bisync for two-way sync rclone bisync /mnt/local-edgar edgar-s3:my-edgar-bucket \ --resync \ --verbose` * * * Performance Optimization ------------------------ ### Caching Recommendations | Scenario | Tool | Cache Settings | | --- | --- | --- | | Frequent reads | goofys | `--stat-cache-ttl 1h` | | Large file writes | rclone | `--vfs-cache-mode full --vfs-cache-max-size 10G` | | Mixed workload | s3fs | `-o use_cache=/tmp/s3cache -o ensure_diskfree=2048` | ### Compression Filings are already compressed by EdgarTools. Additional S3 compression isn't necessary. ### Lifecycle Policies Reduce storage costs with lifecycle rules: `{ "Rules": [ { "ID": "MoveToIA", "Status": "Enabled", "Filter": {"Prefix": "filings/"}, "Transitions": [ { "Days": 30, "StorageClass": "STANDARD_IA" }, { "Days": 180, "StorageClass": "GLACIER" } ] } ] }` * * * Troubleshooting --------------- ### Common Issues **"Transport endpoint not connected"** `# FUSE mount crashed - remount sudo fusermount -u /mnt/edgar-data goofys my-edgar-bucket /mnt/edgar-data` **Slow performance with s3fs** `# Enable parallel requests and caching s3fs bucket /mnt/data \ -o parallel_count=20 \ -o multipart_size=52 \ -o use_cache=/tmp/s3cache \ -o max_stat_cache_size=100000` **Permission denied on mount** `# Add user_allow_other to /etc/fuse.conf echo "user_allow_other" | sudo tee -a /etc/fuse.conf # Mount with allow_other s3fs bucket /mnt/data -o allow_other` **R2 connection issues** `# Ensure region is set to 'auto' s3fs bucket /mnt/data \ -o url=https://ACCOUNT_ID.r2.cloudflarestorage.com \ -o use_path_request_style \ -o sigv2` ### Debugging `# s3fs debug mode s3fs bucket /mnt/data -d -f -o dbglevel=info # rclone debug rclone mount remote:bucket /mnt/data -vv --log-file=/tmp/rclone.log # Check mount status mount | grep fuse df -h /mnt/edgar-data` * * * Security Best Practices ----------------------- ### IAM Policies (AWS) Least-privilege policy for EdgarTools: `{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:GetObject", "s3:PutObject", "s3:ListBucket" ], "Resource": [ "arn:aws:s3:::my-edgar-bucket", "arn:aws:s3:::my-edgar-bucket/*" ] } ] }` ### Encryption `# Enable server-side encryption aws s3api put-bucket-encryption \ --bucket my-edgar-bucket \ --server-side-encryption-configuration \ '{"Rules":[{"ApplyServerSideEncryptionByDefault":{"SSEAlgorithm":"AES256"}}]}'` ### Private Access (No Public URLs) For internal-only access, skip the static website hosting and use: 1. FUSE mount with IAM credentials 2. VPC endpoints for AWS 3. Private connectivity for GCP/Azure * * * Native Cloud Support -------------------- EdgarTools provides native cloud storage support via `fsspec`, enabling seamless integration with S3, Google Cloud Storage, Azure Blob Storage, and S3-compatible services. ### Installation Install the cloud storage dependencies for your provider: `# AWS S3, Cloudflare R2, MinIO, DigitalOcean Spaces pip install "edgartools[s3]" # Google Cloud Storage pip install "edgartools[gcs]" # Azure Blob Storage pip install "edgartools[azure]" # All cloud providers pip install "edgartools[all-cloud]"` ### Basic Usage `import edgar # AWS S3 (uses default credentials from ~/.aws or environment) edgar.use_cloud_storage('s3://my-edgar-bucket/') # Now all operations use cloud storage company = edgar.Company("AAPL") filings = company.get_filings(form="10-K")` ### Provider Examples #### AWS S3 `import edgar # Using default AWS credentials edgar.use_cloud_storage('s3://my-edgar-bucket/') # With explicit credentials edgar.use_cloud_storage( 's3://my-edgar-bucket/', client_kwargs={ 'aws_access_key_id': 'YOUR_ACCESS_KEY', 'aws_secret_access_key': 'YOUR_SECRET_KEY', 'region_name': 'us-east-1' } )` #### Cloudflare R2 `import edgar edgar.use_cloud_storage( 's3://my-bucket/', client_kwargs={ 'endpoint_url': 'https://ACCOUNT_ID.r2.cloudflarestorage.com', 'region_name': 'auto' } )` #### Google Cloud Storage `import edgar # Using default GCP credentials edgar.use_cloud_storage('gs://my-edgar-bucket/') # With explicit project edgar.use_cloud_storage( 'gs://my-edgar-bucket/', client_kwargs={'project': 'my-project'} )` #### Azure Blob Storage `import edgar edgar.use_cloud_storage( 'az://my-container/edgar/', client_kwargs={ 'account_name': 'myaccount', 'account_key': 'YOUR_ACCOUNT_KEY' } )` #### MinIO (Self-Hosted S3) `import edgar edgar.use_cloud_storage( 's3://edgar-data/', client_kwargs={ 'endpoint_url': 'http://localhost:9000', 'aws_access_key_id': 'minioadmin', 'aws_secret_access_key': 'minioadmin' } )` ### Connection Verification By default, `use_cloud_storage()` verifies the connection by listing the bucket. This catches configuration errors early: `import edgar # Fails immediately if credentials are wrong or bucket doesn't exist edgar.use_cloud_storage('s3://my-bucket/') # Skip verification for faster startup (not recommended) edgar.use_cloud_storage('s3://my-bucket/', verify=False)` ### Disabling Cloud Storage `import edgar # Revert to local storage edgar.use_cloud_storage(disable=True)` ### Uploading Data to Cloud Storage EdgarTools provides two ways to populate your cloud storage with SEC data: #### Option 1: Download and Upload in One Step Use the `upload_to_cloud` parameter with `download_filings()`: `import edgar # Configure cloud storage first edgar.use_cloud_storage('s3://my-edgar-bucket/') # Download filings and upload to cloud automatically edgar.download_filings('2025-01-15', upload_to_cloud=True) # Download a date range edgar.download_filings('2025-01-01:2025-01-15', upload_to_cloud=True)` #### Option 2: Sync Existing Local Data Use `sync_to_cloud()` to upload data you've already downloaded locally: `import edgar # Configure cloud storage edgar.use_cloud_storage('s3://my-edgar-bucket/') # Sync all local filings to cloud result = edgar.sync_to_cloud('filings') print(f"Uploaded: {result['uploaded']}, Skipped: {result['skipped']}") # Sync specific date directory edgar.sync_to_cloud('filings/20250115') # Preview what would be uploaded (dry run) edgar.sync_to_cloud('filings', dry_run=True) # Overwrite existing files in cloud edgar.sync_to_cloud('filings', overwrite=True)` #### sync\_to\_cloud() Parameters | Parameter | Type | Default | Description | | --- | --- | --- | --- | | `source_path` | str | None | Subdirectory to sync (e.g., 'filings', 'filings/20250115') | | `pattern` | str | '\*_/_' | Glob pattern for files to sync | | `batch_size` | int | 20 | Number of concurrent uploads | | `overwrite` | bool | False | Overwrite existing files in cloud | | `dry_run` | bool | False | Preview without uploading | #### Return Value `sync_to_cloud()` returns a dict with upload statistics: `{ 'uploaded': 150, # Files successfully uploaded 'skipped': 50, # Files already in cloud (when overwrite=False) 'failed': 0, # Files that failed to upload 'errors': [] # Error messages for failed uploads }` ### Features | Feature | Description | | --- | --- | | **Cross-platform** | Works on Windows, macOS, and Linux | | **No FUSE required** | Pure Python implementation | | **Transparent compression** | Handles `.gz` files automatically | | **Full read/write** | Both reading and writing supported | | **Provider agnostic** | Same API for all cloud providers | * * * Summary ------- | Use Case | Recommended Approach | | --- | --- | | **Native cloud support** | `use_cloud_storage()` (recommended) | | **Read-only HTTP access** | `EDGAR_DATA_URL` + static website | | **Legacy FUSE mount** | goofys or rclone mount | | **On-premises** | MinIO + `use_cloud_storage()` | | **Zero egress costs** | Cloudflare R2 | ### Quick Start **Native cloud storage (recommended):** `import edgar # Install: pip install "edgartools[s3]" edgar.use_cloud_storage('s3://my-edgar-bucket/') # Read from cloud company = edgar.Company("AAPL") # Write to cloud edgar.download_filings('2025-01-15', upload_to_cloud=True) # Or sync existing local data edgar.sync_to_cloud('filings')` **Read-only via HTTP:** `import os os.environ['EDGAR_DATA_URL'] = 'https://your-bucket.s3.amazonaws.com/' os.environ['EDGAR_USE_LOCAL_DATA'] = '1'` **Legacy FUSE mount (Linux/macOS):** `goofys my-edgar-bucket /mnt/edgar-data export EDGAR_LOCAL_DATA_DIR=/mnt/edgar-data export EDGAR_USE_LOCAL_DATA=1` For questions or feedback, see [Discussion #507](https://github.com/dgunning/edgartools/discussions/507) . Back to top --- # XBRL - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/api/xbrl/#xbrl-api-reference) XBRL API Reference ================== The XBRL module provides comprehensive parsing and processing of XBRL (eXtensible Business Reporting Language) data from SEC filings. It includes support for statement standardization, multi-period analysis, and advanced querying capabilities. Module Overview --------------- The XBRL module is organized into several key components: * **Core Classes**: `XBRL`, `XBRLS` for parsing and managing XBRL documents * **Statement Processing**: `Statements`, `Statement` for working with financial statements * **Facts Querying**: `FactsView`, `FactQuery` for querying XBRL facts * **Multi-Period Analysis**: `StitchedStatements`, `StitchedStatement` for comparative analysis * **Standardization**: `StandardConcept` for normalizing company-specific concepts * **Rendering**: `RenderedStatement` for formatted output Core Classes ------------ ### XBRL The main class for parsing and working with XBRL documents from SEC filings. `from edgar.xbrl import XBRL class XBRL: """Main XBRL parser integrating all components of the XBRL parsing system."""` #### Factory Methods #### from\_filing() `@classmethod def from_filing(cls, filing: Filing) -> XBRL` Create an XBRL instance from a Filing object. **Parameters:** - `filing`: SEC filing object containing XBRL data **Returns:** `XBRL` instance **Example:** `from edgar import Company from edgar.xbrl import XBRL company = Company("AAPL") filing = company.latest("10-K") xbrl = XBRL.from_filing(filing)` #### from\_directory() `@classmethod def from_directory(cls, directory: str) -> XBRL` Create an XBRL instance from a directory containing XBRL files. **Parameters:** - `directory`: Path to directory containing XBRL files **Returns:** `XBRL` instance #### from\_files() `@classmethod def from_files(cls, files: List[str]) -> XBRL` Create an XBRL instance from a list of XBRL files. **Parameters:** - `files`: List of file paths to XBRL documents **Returns:** `XBRL` instance #### Core Properties #### statements `@property def statements(self) -> Statements` Access to all financial statements in the XBRL document. **Returns:** `Statements` object for accessing individual statements **Example:** `# Access different statement types balance_sheet = xbrl.statements.balance_sheet() income_statement = xbrl.statements.income_statement() cash_flow = xbrl.statements.cash_flow_statement()` #### facts `@property def facts(self) -> FactsView` Access to all XBRL facts with querying capabilities. **Returns:** `FactsView` object for querying facts **Example:** `# Query facts by concept revenue_facts = xbrl.facts.by_concept("Revenue") # Convert to DataFrame for analysis facts_df = xbrl.facts.to_dataframe()` #### Statement Methods #### get\_statement() `def get_statement(self, statement_type: str) -> Optional[Statement]` Get a specific financial statement by type. **Parameters:** - `statement_type`: Statement type ("BalanceSheet", "IncomeStatement", "CashFlowStatement", etc.) **Returns:** `Statement` object or None if not found #### render\_statement() `def render_statement(self, statement_type: str, **kwargs) -> RenderedStatement` Render a financial statement with rich formatting. **Parameters:** - `statement_type`: Statement type to render - `**kwargs`: Additional rendering options **Returns:** `RenderedStatement` object **Example:** `# Render balance sheet rendered = xbrl.render_statement("BalanceSheet") print(rendered) # Render with custom options rendered = xbrl.render_statement("IncomeStatement", show_percentages=True, max_rows=50)` #### Data Conversion #### to\_pandas() `def to_pandas(self) -> pd.DataFrame` Convert XBRL facts to a pandas DataFrame. **Returns:** DataFrame with all facts and their attributes **Example:** `# Convert to DataFrame for analysis df = xbrl.to_pandas() print(df.columns) # ['concept', 'value', 'period', 'label', ...] # Filter for specific concepts revenue_df = df[df['concept'].str.contains('Revenue', case=False)]` ### XBRLS Container class for managing multiple XBRL documents for multi-period analysis. `from edgar.xbrl import XBRLS class XBRLS: """Container for multiple XBRL objects enabling multi-period analysis."""` #### Factory Methods #### from\_filings() `@classmethod def from_filings(cls, filings: List[Filing]) -> XBRLS` Create an XBRLS instance from multiple filings. **Parameters:** - `filings`: List of Filing objects **Returns:** `XBRLS` instance **Example:** `from edgar import Company from edgar.xbrl import XBRLS company = Company("AAPL") filings = company.get_filings(form="10-K").head(3) # Get 3 years xbrls = XBRLS.from_filings(filings)` #### Properties #### statements `@property def statements(self) -> StitchedStatements` Access to stitched statements showing multi-period data. **Returns:** `StitchedStatements` object **Example:** `# Get multi-period statements income_stmt = xbrls.statements.income_statement() balance_sheet = xbrls.statements.balance_sheet() # Render multi-period view print(income_stmt.render())` Statement Classes ----------------- ### Statements High-level interface for accessing financial statements from a single XBRL document. `class Statements: """High-level interface to all statements in an XBRL document."""` #### Statement Access Methods #### balance\_sheet() `def balance_sheet(self) -> Optional[Statement]` Get the balance sheet statement. **Returns:** `Statement` object or None #### income\_statement() `def income_statement(self) -> Optional[Statement]` Get the income statement. **Returns:** `Statement` object or None #### cash\_flow\_statement() `def cash_flow_statement(self) -> Optional[Statement]` Get the cash flow statement. **Returns:** `Statement` object or None #### statement\_of\_equity() `def statement_of_equity(self) -> Optional[Statement]` Get the statement of equity. **Returns:** `Statement` object or None #### comprehensive\_income() `def comprehensive_income(self) -> Optional[Statement]` Get the comprehensive income statement. **Returns:** `Statement` object or None **Example:** `statements = xbrl.statements # Access different statement types if statements.balance_sheet(): bs = statements.balance_sheet() print(f"Total Assets: {bs.get_concept_value('Assets')}") if statements.income_statement(): is_stmt = statements.income_statement() print(f"Revenue: {is_stmt.get_concept_value('Revenue')}")` ### Statement Individual financial statement with analysis and rendering capabilities. `class Statement: """A single financial statement extracted from XBRL data."""` #### Core Methods #### render() `def render(self, **kwargs) -> RenderedStatement` Render the statement with rich formatting. **Parameters:** - `**kwargs`: Rendering options (show\_percentages, max\_rows, etc.) **Returns:** `RenderedStatement` object #### to\_dataframe() `def to_dataframe( self, include_dimensions: bool = True, include_unit: bool = False, include_point_in_time: bool = False, presentation: bool = False ) -> pd.DataFrame` Convert statement to pandas DataFrame with optional transformations. **Parameters:** - `include_dimensions`: Include dimensional breakdowns (default: True) - `include_unit`: Include unit column (USD, shares, etc.) (default: False) - `include_point_in_time`: Include point-in-time column for instant facts (default: False) - `presentation`: Apply HTML-matching transformations using preferred\_sign (default: False) - False (default): Raw instance values from XML - True: Transform values to match SEC filing HTML display **Returns:** DataFrame with the following columns: - **Core columns**: `concept`, `label`, period columns (dates) - **Metadata columns** (always included): - `balance` — debit or credit (from XBRL taxonomy) - `weight` — calculation tree weight (+1 or -1) - `preferred_sign` — how the value should be displayed (from presentation linkbase) - `level` — nesting depth in the presentation tree (0=root, 1=section header, 2=line item, etc.) - `abstract` — True if this row is a section header, not a data row - `parent_concept` — calculation tree parent (the metric concept this rolls up to for summation math) - `parent_abstract_concept` — presentation tree parent (the section header this appears under for display hierarchy) - **Optional columns**: `dimension`, `unit`, `point_in_time` **Value Modes:** - **Raw mode** (default): Preserves values exactly as reported in instance document - **Presentation mode** (`presentation=True`): Applies transformations to match SEC HTML rendering - Cash Flow: outflows with preferred\_sign=-1 shown as negative - Income Statement: applies preferred\_sign transformations **Example:** `statement = xbrl.statements.income_statement() # Raw values (default) df_raw = statement.to_dataframe() # Returns actual XML values + metadata columns # Presentation mode (matches SEC HTML) df_presentation = statement.to_dataframe(presentation=True) # Returns transformed values matching 10-K HTML display # Check metadata print(df_raw[['concept', 'balance', 'weight', 'preferred_sign']].head()) # Hierarchy columns — understand parent-child relationships print(df_raw[['label', 'level', 'parent_concept', 'parent_abstract_concept']].head(10))` **See Also:** - Issue #463 - XBRL value transformations and metadata columns - Issue #514 - Parent concept hierarchy columns - [Revenue Segment Hierarchy Guide](https://edgartools.readthedocs.io/en/stable/guides/extract-statements/#understanding-statement-hierarchy) **Returns:** DataFrame with statement data #### get\_concept\_value() `def get_concept_value(self, concept: str) -> Optional[Any]` Get the value for a specific concept. **Parameters:** - `concept`: Concept name to look up **Returns:** Concept value or None **Example:** `statement = xbrl.statements.income_statement() # Render the statement rendered = statement.render() print(rendered) # Convert to DataFrame df = statement.to_dataframe() # Get specific values revenue = statement.get_concept_value("Revenue") net_income = statement.get_concept_value("NetIncomeLoss")` Facts Querying -------------- ### FactsView Provides a view over all XBRL facts with analysis and querying methods. `class FactsView: """View over all facts with analysis methods."""` #### Query Methods #### by\_concept() `def by_concept(self, pattern: str, exact: bool = False) -> FactQuery` Filter facts by concept name. **Parameters:** - `pattern`: Pattern to match against concept names - `exact`: If True, require exact match; otherwise, use regex **Returns:** `FactQuery` object for further filtering #### by\_label() `def by_label(self, pattern: str, exact: bool = False) -> FactQuery` Filter facts by element label. **Parameters:** - `pattern`: Pattern to match against labels - `exact`: If True, require exact match; otherwise, use regex **Returns:** `FactQuery` object for further filtering #### by\_value() `def by_value(self, min_value: float = None, max_value: float = None) -> FactQuery` Filter facts by value range. **Parameters:** - `min_value`: Minimum value threshold - `max_value`: Maximum value threshold **Returns:** `FactQuery` object for further filtering #### by\_period() `def by_period(self, start_date: str = None, end_date: str = None) -> FactQuery` Filter facts by period range. **Parameters:** - `start_date`: Start date (YYYY-MM-DD format) - `end_date`: End date (YYYY-MM-DD format) **Returns:** `FactQuery` object for further filtering #### Analysis Methods #### pivot\_by\_period() `def pivot_by_period(self, concepts: List[str] = None) -> pd.DataFrame` Create a pivot table showing concepts by period. **Parameters:** - `concepts`: List of concepts to include (default: all) **Returns:** DataFrame with concepts as rows and periods as columns #### time\_series() `def time_series(self, concept: str) -> pd.Series` Get time series data for a specific concept. **Parameters:** - `concept`: Concept name **Returns:** pandas Series with time series data #### Data Conversion #### to\_dataframe() `def to_dataframe(self) -> pd.DataFrame` Convert facts to pandas DataFrame. **Returns:** DataFrame with all facts and metadata **Example:** `facts = xbrl.facts # Query by concept revenue_query = facts.by_concept("Revenue") revenue_facts = revenue_query.execute() # Query by label and value large_expenses = facts.by_label("expense").by_value(min_value=1000000) expense_facts = large_expenses.to_dataframe() # Time series analysis revenue_ts = facts.time_series("Revenue") print(revenue_ts.head()) # Pivot analysis pivot_df = facts.pivot_by_period(["Revenue", "NetIncomeLoss"])` ### FactQuery Fluent query builder for filtering and manipulating XBRL facts. `class FactQuery: """A query builder for XBRL facts with fluent interface."""` #### Filtering Methods All filtering methods return `self` for method chaining. #### by\_concept() `def by_concept(self, pattern: str, exact: bool = False) -> FactQuery` #### by\_label() `def by_label(self, pattern: str, exact: bool = False) -> FactQuery` #### by\_value() `def by_value(self, min_value: float = None, max_value: float = None) -> FactQuery` #### by\_period() `def by_period(self, start_date: str = None, end_date: str = None) -> FactQuery` #### by\_statement() `def by_statement(self, statement_type: str) -> FactQuery` Filter facts by statement type. **Parameters:** - `statement_type`: Statement type to filter by **Returns:** `FactQuery` object for method chaining #### Execution Methods #### execute() `def execute(self) -> List[Dict]` Execute the query and return matching facts. **Returns:** List of fact dictionaries #### to\_dataframe() `def to_dataframe(self) -> pd.DataFrame` Execute the query and return results as DataFrame. **Returns:** DataFrame with query results #### first() `def first(self) -> Optional[Dict]` Get the first matching fact. **Returns:** First fact dictionary or None #### count() `def count(self) -> int` Count matching facts without retrieving them. **Returns:** Number of matching facts **Example:** `# Chain multiple filters query = (xbrl.facts .by_concept("Revenue") .by_period(start_date="2023-01-01") .by_value(min_value=1000000)) # Execute in different ways facts_list = query.execute() facts_df = query.to_dataframe() first_fact = query.first() count = query.count()` Multi-Period Analysis --------------------- ### StitchedStatements Interface for accessing multi-period statements that combine data across multiple XBRL documents. `class StitchedStatements: """Interface for multi-period statements."""` #### Statement Access Methods Similar to `Statements` but returns `StitchedStatement` objects. All methods accept these common parameters: * `max_periods` (int): Maximum number of periods to include (default: 8) * `standard` (bool): Whether to use standardized concept labels (default: True) * `use_optimal_periods` (bool): Whether to use entity info for optimal period selection (default: True) * `show_date_range` (bool): Whether to show full date ranges for duration periods (default: False) * `include_dimensions` (bool): Whether to include dimensional segment data (default: False, True for equity/comprehensive income) * `view` (str): Controls dimensional filtering — `"standard"`, `"detailed"`, or `"summary"`. Overrides `include_dimensions` when provided. #### balance\_sheet() `def balance_sheet(self, view=None, **kwargs) -> Optional[StitchedStatement]` #### income\_statement() `def income_statement(self, view=None, **kwargs) -> Optional[StitchedStatement]` #### cashflow\_statement() `def cashflow_statement(self, view=None, **kwargs) -> Optional[StitchedStatement]` #### statement\_of\_equity() `def statement_of_equity(self, view=None, **kwargs) -> Optional[StitchedStatement]` #### comprehensive\_income() `def comprehensive_income(self, view=None, **kwargs) -> Optional[StitchedStatement]` **Example:** `# Multi-period analysis stitched_statements = xbrls.statements income_stmt = stitched_statements.income_statement() # Shows multiple years of data print(income_stmt.render()) # Include dimensional breakdowns (e.g., cost by segment) income_detailed = stitched_statements.income_statement(view="detailed") df = income_detailed.to_dataframe()` ### StitchedStatement Individual statement showing multi-period data with comparative analysis. `class StitchedStatement: """Individual stitched statement showing multi-period data."""` **Constructor Parameters:** - `xbrls`: XBRLS object containing stitched data - `statement_type` (str): Type of statement ('BalanceSheet', 'IncomeStatement', etc.) - `max_periods` (int): Maximum number of periods (default: 8) - `standard` (bool): Use standardized labels (default: True) - `include_dimensions` (bool): Include dimensional data (default: False) - `view` (str): `"standard"`, `"detailed"`, or `"summary"`. Overrides `include_dimensions`. #### Analysis Methods #### render() `def render(self, show_date_range: bool = False) -> Table` Render multi-period statement with rich formatting. #### to\_dataframe() `def to_dataframe(self) -> pd.DataFrame` Convert to DataFrame with periods as columns. Standardization --------------- ### StandardConcept Represents a standardized concept that normalizes company-specific terminology. `class StandardConcept: """Standardized concept representation."""` #### Properties #### name `@property def name(self) -> str` Standardized concept name. #### label `@property def label(self) -> str` Standardized human-readable label. **Example:** `# Standardization is applied automatically in statements statement = xbrl.statements.income_statement() df = statement.to_dataframe() # Check for standardized vs original labels print(df[['label', 'original_label']].head())` Rendering --------- ### RenderedStatement Formatted statement output with rich console display capabilities. `class RenderedStatement: """Rich formatted statement output."""` #### Display Methods #### **str**() `def __str__(self) -> str` Plain text representation of the statement. #### **rich**() `def __rich__(self) -> RichRenderable` Rich console representation with formatting. **Example:** `# Rich rendering in console rendered = xbrl.render_statement("BalanceSheet") print(rendered) # Displays with rich formatting # Plain text for export text_output = str(rendered)` Utility Functions ----------------- ### stitch\_statements() `def stitch_statements(statements: List[Statement]) -> StitchedStatement` Combine multiple statements into a stitched statement. **Parameters:** - `statements`: List of Statement objects to combine **Returns:** `StitchedStatement` object ### render\_stitched\_statement() `def render_stitched_statement(stitched_statement: StitchedStatement, **kwargs) -> RenderedStatement` Render a stitched statement with formatting. **Parameters:** - `stitched_statement`: StitchedStatement to render - `**kwargs`: Rendering options **Returns:** `RenderedStatement` object ### to\_pandas() `def to_pandas(obj: Union[XBRL, Statement, FactsView]) -> pd.DataFrame` Convert various XBRL objects to pandas DataFrame. **Parameters:** - `obj`: Object to convert (XBRL, Statement, or FactsView) **Returns:** DataFrame representation Advanced Usage Examples ----------------------- ### Multi-Period Financial Analysis `from edgar import Company from edgar.xbrl import XBRLS # Get multiple years of data company = Company("AAPL") filings = company.get_filings(form="10-K").head(3) xbrls = XBRLS.from_filings(filings) # Analyze income statement trends income_stmt = xbrls.statements.income_statement() revenue_trend = income_stmt.get_trend("Revenue") revenue_growth = income_stmt.calculate_growth("Revenue") print(f"Revenue Growth: {revenue_growth.iloc[-1]:.2%}")` ### Complex Fact Querying `from edgar import Company from edgar.xbrl import XBRL company = Company("MSFT") filing = company.latest("10-K") xbrl = XBRL.from_filing(filing) # Complex query with multiple filters high_value_revenue = (xbrl.facts .by_concept("Revenue") .by_value(min_value=50000000000) # $50B+ .by_period(start_date="2023-01-01") .to_dataframe()) # Pivot analysis pivot_df = xbrl.facts.pivot_by_period([ "Revenue", "NetIncomeLoss", "OperatingIncomeLoss" ])` ### Statement Comparison `# Compare statements across different companies companies = ["AAPL", "MSFT", "GOOGL"] statements = [] for ticker in companies: company = Company(ticker) filing = company.latest("10-K") xbrl = XBRL.from_filing(filing) if xbrl.statements.income_statement(): statements.append(xbrl.statements.income_statement()) # Create comparison DataFrame comparison_data = [] for stmt in statements: df = stmt.to_dataframe() comparison_data.append(df) # Analyze key metrics across companies key_metrics = ["Revenue", "NetIncomeLoss", "OperatingIncomeLoss"] for metric in key_metrics: print(f"\n{metric} Comparison:") for i, stmt in enumerate(statements): value = stmt.get_concept_value(metric) if value: print(f" {companies[i]}: ${value/1e9:.1f}B")` Import Reference ---------------- `# Core classes from edgar.xbrl import XBRL, XBRLS # Statement classes from edgar.xbrl import Statements, Statement from edgar.xbrl import StitchedStatements, StitchedStatement # Facts querying from edgar.xbrl import FactsView, FactQuery from edgar.xbrl import StitchedFactsView, StitchedFactQuery # Standardization and rendering from edgar.xbrl import StandardConcept, RenderedStatement # Utility functions from edgar.xbrl import stitch_statements, render_stitched_statement, to_pandas` Error Handling -------------- `from edgar.xbrl import XBRL, XBRLFilingWithNoXbrlData try: xbrl = XBRL.from_filing(filing) except XBRLFilingWithNoXbrlData: print("Filing does not contain XBRL data") except Exception as e: print(f"Error parsing XBRL: {e}") # Check for statement availability if xbrl.statements.income_statement(): income_stmt = xbrl.statements.income_statement() df = income_stmt.to_dataframe() else: print("Income statement not found")` XBRL Value Transformations (Issue #463) --------------------------------------- EdgarTools provides a two-layer system for XBRL value handling: ### Value Layers 1. **Raw Values** (default): Values exactly as reported in the XBRL instance document 2. Matches SEC CompanyFacts API 3. Preserves original data for analysis 4. No transformations applied 5. **Presentation Values** (`presentation=True`): Values transformed to match SEC filing HTML display 6. Applies `preferred_sign` transformations from presentation linkbase 7. Cash Flow outflows shown as negative when appropriate 8. Matches how values appear in the official 10-K/10-Q HTML ### Metadata Columns All statement DataFrames include XBRL metadata columns: * **`balance`**: Debit or credit classification from schema (accounting semantics) * **`weight`**: Calculation weight from calculation linkbase (+1.0 or -1.0) * **`preferred_sign`**: Presentation hint from presentation linkbase (+1 or -1) These columns provide transparency about XBRL semantics and enable custom transformations. ### Usage Examples `# Get raw values (default) xbrl = filing.xbrl() statement = xbrl.statements.cash_flow_statement() df_raw = statement.to_dataframe() # PaymentsOfDividends appears as positive (raw XML value) dividends = df_raw[df_raw['concept'].str.contains('PaymentsOfDividends')] print(dividends[['concept', 'balance', 'preferred_sign', '2024-09-30']]) # Output: concept=PaymentsOfDividends, balance=credit, preferred_sign=-1, value=12345000000 (positive) # Get presentation values (matches SEC HTML) df_presentation = statement.to_dataframe(presentation=True) dividends_pres = df_presentation[df_presentation['concept'].str.contains('PaymentsOfDividends')] print(dividends_pres[['concept', '2024-09-30']]) # Output: value=-12345000000 (negative, matches HTML display with parentheses)` ### When to Use Each Mode **Use Raw Values** (default): - Cross-company financial analysis - Data science and machine learning - Comparison with SEC CompanyFacts API - When you need unmodified reported values **Use Presentation Values** (`presentation=True`): - Matching SEC filing HTML display - Creating investor-facing reports - Replicating official financial statement appearance - When users expect "traditional" financial statement signs ### Technical Notes * **Raw values are consistent across companies**: Testing confirmed SEC instance data uses consistent signs * **Metadata always included**: All transformations can be recreated using metadata columns * **No data loss**: Raw values always preserved, transformations are reversible Performance Tips ---------------- 1. **Use specific queries** - Filter facts early to reduce processing time 2. **Cache XBRL objects** - Parsing is expensive, reuse when possible 3. **Limit statement rendering** - Use `max_rows` parameter for large statements 4. **Batch processing** - Use `XBRLS` for efficient multi-period analysis See Also -------- * **[Company API Reference](https://edgartools.readthedocs.io/en/stable/api/company/) ** - Working with company data * **[Filing API Reference](https://edgartools.readthedocs.io/en/stable/api/filing/) ** - Working with individual filings * **[Extract Financial Statements Guide](https://edgartools.readthedocs.io/en/stable/guides/extract-statements/) ** - Practical examples * **[Working with Filing Guide](https://edgartools.readthedocs.io/en/stable/guides/working-with-filing/) ** - Filing workflows Back to top --- # Entity Facts - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/api/entity-facts-reference/#entityfacts-api-reference) EntityFacts API Reference ========================= Complete API documentation for the enhanced EntityFacts system, including all classes, methods, and data models. Overview -------- The EntityFacts API provides structured access to SEC company financial data with AI-ready features, powerful querying capabilities, and professional formatting. The system consists of several key components: * **EntityFacts** - Main class for accessing company facts * **FactQuery** - Fluent query builder for advanced filtering * **FinancialStatement** - Formatted display wrapper for financial data * **FinancialFact** - Individual fact data model with rich metadata EntityFacts Class ----------------- The main entry point for accessing company financial facts. ### Constructor `EntityFacts(cik: int, name: str, facts: List[FinancialFact])` **Parameters:** - `cik` (int): Company CIK number - `name` (str): Company name \- `facts` (List\[FinancialFact\]): List of financial facts ### Properties #### Core Properties #### `cik: int` The company's CIK (Central Index Key) number. `facts = company.facts print(facts.cik) # 320193` #### `name: str` The company's official name. `facts = company.facts print(facts.name) # "Apple Inc."` #### DEI Properties #### `shares_outstanding: Optional[float]` Number of common shares outstanding. `shares = facts.shares_outstanding if shares: print(f"Shares Outstanding: {shares:,.0f}")` #### `public_float: Optional[float]` Public float value in dollars. `float_val = facts.public_float if float_val: print(f"Public Float: ${float_val:,.0f}")` #### `shares_outstanding_fact: Optional[FinancialFact]` Full fact object for shares outstanding with metadata. `fact = facts.shares_outstanding_fact if fact: print(f"Shares: {fact.get_formatted_value()} as of {fact.period_end}")` #### `public_float_fact: Optional[FinancialFact]` Full fact object for public float with metadata. `fact = facts.public_float_fact if fact: print(f"Float: {fact.get_formatted_value()} as of {fact.period_end}")` ### Core Methods #### Query Interface #### `query() -> FactQuery` Start building a facts query using the fluent interface. `query = facts.query() results = query.by_concept('Revenue').latest(4)` **Returns:** FactQuery builder instance #### `get_fact(concept: str, period: Optional[str] = None) -> Optional[FinancialFact]` Get a single fact by concept name. `# Use the full XBRL concept name or the lowercase label revenue_fact = facts.get_fact('us-gaap:Revenues') q1_revenue = facts.get_fact('us-gaap:Revenues', '2024-Q1') # Lowercase labels also work revenue_fact = facts.get_fact('revenues')` **Parameters:** - `concept` (str): XBRL concept name (e.g. `'us-gaap:Revenues'`) or lowercase label (e.g. `'revenues'`). Use `search_concepts()` to find valid names. - `period` (str, optional): Period in format "YYYY-QN" or "YYYY-FY" **Returns:** Most recent matching fact or None `time_series(concept: str, periods: int = 20) -> pd.DataFrame` Get time series data for a concept. `revenue_ts = facts.time_series('Revenue', periods=8)` **Parameters:** - `concept` (str): Concept name or label - `periods` (int): Number of periods to retrieve (default: 20) **Returns:** DataFrame with time series data ### Financial Statement Methods `income_statement(periods: int = 4, period_length: Optional[int] = None, as_dataframe: bool = False, annual: bool = True)` Get income statement facts formatted as a financial statement. `# Default: 4 annual periods, formatted display stmt = facts.income_statement() # 8 quarterly periods as DataFrame df = facts.income_statement(periods=8, annual=False, as_dataframe=True)` **Parameters:** - `periods` (int): Number of periods to retrieve (default: 4) - `period_length` (int, optional): Filter by period length in months (3=quarterly, 12=annual) - `as_dataframe` (bool): If True, return DataFrame; if False, return FinancialStatement (default: False) - `annual` (bool): If True, prefer annual periods; if False, prefer quarterly (default: True) **Returns:** FinancialStatement or DataFrame #### `balance_sheet(periods: int = 4, as_of: Optional[date] = None, as_dataframe: bool = False, annual: bool = True)` Get balance sheet facts for periods or point-in-time. `# Multi-period balance sheet stmt = facts.balance_sheet(periods=4) # Point-in-time snapshot snapshot = facts.balance_sheet(as_of=date(2024, 12, 31))` **Parameters:** - `periods` (int): Number of periods to retrieve (default: 4) - `as_of` (date, optional): Get snapshot as of specific date - `as_dataframe` (bool): If True, return DataFrame; if False, return FinancialStatement (default: False) - `annual` (bool): If True, prefer annual periods (default: True) **Returns:** FinancialStatement or DataFrame #### `cash_flow(periods: int = 4, period_length: Optional[int] = None, as_dataframe: bool = False, annual: bool = True)` Get cash flow statement facts. `# Annual cash flow trends stmt = facts.cashflow_statement(periods=5, annual=True)` **Parameters:** - `periods` (int): Number of periods to retrieve (default: 4) - `period_length` (int, optional): Filter by period length in months - `as_dataframe` (bool): If True, return DataFrame; if False, return FinancialStatement (default: False) - `annual` (bool): If True, prefer annual periods (default: True) **Returns:** FinancialStatement or DataFrame ### DEI Methods #### `dei_facts(as_of: Optional[date] = None) -> pd.DataFrame` Get Document and Entity Information facts. `# Latest DEI facts dei = facts.dei_facts() # DEI facts as of specific date dei = facts.dei_facts(as_of=date(2024, 12, 31))` **Parameters:** - `as_of` (date, optional): Get facts as of specific date **Returns:** DataFrame with DEI facts #### `entity_info() -> Dict[str, Any]` Get key entity information as a clean dictionary. `info = facts.entity_info() print(info['entity_name']) print(info['shares_outstanding'])` **Returns:** Dictionary with entity information ### AI/LLM Methods #### `to_llm_context(focus_areas: Optional[List[str]] = None, time_period: str = "recent") -> Dict[str, Any]` Generate comprehensive context for LLM analysis. `context = facts.to_llm_context( focus_areas=['profitability', 'growth'], time_period='5Y' )` **Parameters:** - `focus_areas` (List\[str\], optional): Areas to emphasize (\['profitability', 'growth', 'liquidity'\]) - `time_period` (str): Time period to analyze ('recent', '5Y', '10Y', 'all') (default: 'recent') **Returns:** Dictionary with structured LLM context #### `to_agent_tools() -> List[Dict[str, Any]]` Export facts as MCP-compatible tools for AI agents. `tools = facts.to_agent_tools()` **Returns:** List of tool definitions ### Magic Methods #### `__len__() -> int` Get total number of facts. `total_facts = len(facts)` #### `__iter__() -> Iterator[FinancialFact]` Iterate over all facts. `for fact in facts: print(f"{fact.concept}: {fact.numeric_value}")` FactQuery Class --------------- Fluent query builder for advanced fact filtering and analysis. ### Constructor Created via `EntityFacts.query()` method. Do not instantiate directly. ### Filtering Methods #### Concept Filtering #### `by_concept(concept: str, exact: bool = False) -> FactQuery` Filter by concept name or pattern. `# Fuzzy matching (default) revenue_facts = query.by_concept('Revenue') # Exact matching exact_revenue = query.by_concept('us-gaap:Revenue', exact=True)` **Parameters:** - `concept` (str): Concept name or label to match - `exact` (bool): If True, require exact match (default: False) #### `by_label(label: str, fuzzy: bool = True) -> FactQuery` Filter by human-readable label. `# Fuzzy label matching facts = query.by_label('Total Revenue', fuzzy=True) # Exact label matching facts = query.by_label('Revenue', fuzzy=False)` **Parameters:** - `label` (str): Label to match - `fuzzy` (bool): Use fuzzy matching (default: True) #### Time-Based Filtering #### `by_fiscal_year(year: int) -> FactQuery` Filter by fiscal year. `fy2024_facts = query.by_fiscal_year(2024)` **Parameters:** - `year` (int): Fiscal year to filter by #### `by_fiscal_period(period: str) -> FactQuery` Filter by fiscal period. `q1_facts = query.by_fiscal_period('Q1') fy_facts = query.by_fiscal_period('FY')` **Parameters:** - `period` (str): Fiscal period ('FY', 'Q1', 'Q2', 'Q3', 'Q4') #### `by_period_length(months: int) -> FactQuery` Filter by period length in months. `# Quarterly periods (3 months) quarterly = query.by_period_length(3) # Annual periods (12 months) annual = query.by_period_length(12)` **Parameters:** - `months` (int): Period length (3=quarterly, 12=annual, 9=YTD) #### `date_range(start: date, end: date) -> FactQuery` Filter by date range. `recent_facts = query.date_range( start=date(2023, 1, 1), end=date(2024, 12, 31) )` **Parameters:** - `start` (date): Start date (inclusive) - `end` (date): End date (inclusive) #### `as_of(as_of_date: date) -> FactQuery` Get facts as of specific date (point-in-time). `snapshot = query.as_of(date(2024, 6, 30))` **Parameters:** - `as_of_date` (date): Date for point-in-time view #### Statement and Form Filtering #### `by_statement_type(statement_type: str) -> FactQuery` Filter by financial statement type. `income_facts = query.by_statement_type('IncomeStatement') balance_facts = query.by_statement_type('BalanceSheet') cash_facts = query.by_statement_type('CashFlow')` **Parameters:** - `statement_type` (str): Statement type ('IncomeStatement', 'BalanceSheet', 'CashFlow') #### `by_form_type(form_type: Union[str, List[str]]) -> FactQuery` Filter by SEC form type. `# Single form type annual_facts = query.by_form_type('10-K') # Multiple form types periodic_facts = query.by_form_type(['10-K', '10-Q'])` **Parameters:** - `form_type` (str or List\[str\]): Form type(s) to filter by #### Quality Filtering #### `high_quality_only() -> FactQuery` Filter to only high-quality, audited facts. `quality_facts = query.high_quality_only()` #### `min_confidence(threshold: float) -> FactQuery` Filter by minimum confidence score. `confident_facts = query.min_confidence(0.9)` **Parameters:** - `threshold` (float): Minimum confidence score (0.0 to 1.0) #### Special Queries #### `latest_instant() -> FactQuery` Filter to most recent instant facts (for balance sheet items). `latest_balance = query.by_statement_type('BalanceSheet').latest_instant()` #### `latest_periods(n: int = 4, annual: bool = True) -> FactQuery` Get facts from the n most recent periods. `# Latest 4 annual periods only recent = query.latest_periods(4, annual=True) # Latest 8 periods, any type recent = query.latest_periods(8, annual=False)` **Parameters:** - `n` (int): Number of recent periods (default: 4) - `annual` (bool): If True, only use annual periods; if False, use all period types (default: True) ### Sorting and Limiting #### `sort_by(field: str, ascending: bool = True) -> FactQuery` Sort results by field. `# Sort by filing date (newest first) sorted_facts = query.sort_by('filing_date', ascending=False) # Sort by fiscal year sorted_facts = query.sort_by('fiscal_year')` **Parameters:** - `field` (str): Field name to sort by - `ascending` (bool): Sort order (default: True) #### `latest(n: int = 1) -> List[FinancialFact]` Get the n most recent facts. `latest_revenue = query.by_concept('Revenue').latest(5)` **Parameters:** - `n` (int): Number of facts to return (default: 1) **Returns:** List of facts (executes query immediately) ### Execution Methods #### `execute() -> List[FinancialFact]` Execute query and return matching facts. `facts = query.by_concept('Revenue').by_fiscal_year(2024).execute()` **Returns:** List of FinancialFact objects #### `count() -> int` Get count of facts matching current filters. `revenue_count = query.by_concept('Revenue').count()` **Returns:** Number of matching facts ### Output Methods #### `to_dataframe(*columns) -> pd.DataFrame` Convert results to pandas DataFrame. `# All columns df = query.by_concept('Revenue').to_dataframe() # Selected columns df = query.by_concept('Revenue').to_dataframe( 'label', 'numeric_value', 'fiscal_period' )` **Parameters:** - `*columns` (str): Optional column names to include **Returns:** DataFrame with query results #### `pivot_by_period(return_statement: bool = True) -> Union[FinancialStatement, pd.DataFrame]` Pivot facts to show concepts as rows and periods as columns. `# Formatted financial statement stmt = query.by_statement_type('IncomeStatement').pivot_by_period() # Raw DataFrame df = query.by_statement_type('IncomeStatement').pivot_by_period(return_statement=False)` **Parameters:** - `return_statement` (bool): If True, return FinancialStatement; if False, return DataFrame (default: True) **Returns:** FinancialStatement or DataFrame #### `to_llm_context() -> List[Dict[str, Any]]` Convert results to LLM-friendly context. `llm_data = query.by_concept('Revenue').to_llm_context()` **Returns:** List of fact contexts for LLM consumption FinancialStatement Class ------------------------ Wrapper around pandas DataFrame for financial statements with intelligent formatting. ### Constructor `FinancialStatement( data: pd.DataFrame, statement_type: str, entity_name: str = "", period_lengths: Optional[List[str]] = None, mixed_periods: bool = False )` **Parameters:** - `data` (pd.DataFrame): Financial data - `statement_type` (str): Statement type - `entity_name` (str): Company name - `period_lengths` (List\[str\], optional): Period lengths in data - `mixed_periods` (bool): Whether data contains mixed period lengths ### Properties #### `shape: tuple` Shape of the underlying DataFrame. `stmt = company.income_statement() print(stmt.shape) # (10, 4)` #### `columns: pd.Index` Column names of the statement. `periods = stmt.columns print(list(periods)) # ['FY 2024', 'FY 2023', 'FY 2022', 'FY 2021']` #### `index: pd.Index` Row labels (concept names). `concepts = stmt.index print(list(concepts)) # ['Revenue', 'Cost of Revenue', 'Gross Profit', ...]` #### `empty: bool` Whether the statement is empty. `if not stmt.empty: print("Statement has data")` ### Methods #### `to_numeric() -> pd.DataFrame` Get underlying numeric DataFrame for calculations. `stmt = company.income_statement() numeric_data = stmt.to_numeric() growth_rates = numeric_data.pct_change(axis=1)` **Returns:** DataFrame with original numeric values #### `get_concept(concept_name: str) -> Optional[pd.Series]` Get data for specific concept across all periods. `revenue_series = stmt.get_concept('Revenue') if revenue_series is not None: print(revenue_series)` **Parameters:** - `concept_name` (str): Name of concept to retrieve **Returns:** Series with values across periods, or None #### `calculate_growth(concept_name: str, periods: int = 2) -> Optional[pd.Series]` Calculate period-over-period growth for a concept. `revenue_growth = stmt.calculate_growth('Revenue', periods=1)` **Parameters:** - `concept_name` (str): Name of concept - `periods` (int): Number of periods for growth calculation (default: 2) **Returns:** Series with growth rates, or None #### `format_value(value: float, concept_label: str) -> str` Format a single value based on its concept. `formatted = stmt.format_value(1234567, 'Revenue') print(formatted) # "$1,234,567"` **Parameters:** - `value` (float): Numeric value to format - `concept_label` (str): Label of financial concept **Returns:** Formatted string #### `to_llm_context() -> Dict[str, Any]` Generate LLM-friendly context from the statement. `context = stmt.to_llm_context()` **Returns:** Dictionary with structured financial data ### Display Methods The FinancialStatement class provides rich display capabilities: * **Jupyter Notebooks**: Automatic HTML rendering with professional styling * **Console**: Formatted text output with proper alignment * **Rich Integration**: Compatible with Rich library for enhanced terminal display FinancialFact Class ------------------- Individual financial fact with rich metadata and AI-ready features. ### Constructor `FinancialFact( concept: str, taxonomy: str, label: str, value: Union[float, int, str], numeric_value: Optional[float], unit: str, scale: Optional[int] = None, # ... additional parameters )` ### Core Attributes #### `concept: str` Standardized concept identifier (e.g., 'us-gaap:Revenue'). #### `taxonomy: str` Taxonomy namespace (us-gaap, ifrs, etc.). #### `label: str` Human-readable label. #### `value: Union[float, int, str]` The actual fact value. #### `numeric_value: Optional[float]` Numeric representation for calculations. #### `unit: str` Unit of measure (USD, shares, etc.). #### `scale: Optional[int]` Scale factor (1000, 1000000, etc.). ### Temporal Attributes #### `period_start: Optional[date]` Period start date (for duration facts). #### `period_end: date` Period end date. #### `period_type: Literal['instant', 'duration']` Type of period. #### `fiscal_year: int` Fiscal year. #### `fiscal_period: str` Fiscal period (FY, Q1, Q2, Q3, Q4). ### Filing Context #### `filing_date: date` Date the fact was filed with SEC. #### `form_type: str` SEC form type (10-K, 10-Q, etc.). #### `accession: str` SEC accession number. ### Quality Indicators #### `data_quality: DataQuality` Data quality enum (HIGH, MEDIUM, LOW). #### `is_audited: bool` Whether the fact is from audited filing. #### `confidence_score: float` Confidence score (0.0 to 1.0). ### AI-Ready Attributes #### `semantic_tags: List[str]` Semantic tags for AI processing. #### `business_context: str` Business context description. ### Methods #### `to_llm_context() -> Dict[str, Any]` Generate rich context for LLM consumption. `fact = facts.get_fact('us-gaap:Revenues') context = fact.to_llm_context() print(context['concept']) print(context['value']) print(context['period'])` **Returns:** Dictionary with formatted context #### `get_formatted_value() -> str` Format the numeric value for display. `fact = facts.get_fact('us-gaap:Revenues') formatted = fact.get_formatted_value() print(formatted) # "365,817,000,000"` **Returns:** Formatted string representation #### `get_display_period_key() -> str` Generate display-friendly period key. `fact = facts.get_fact('us-gaap:Revenues') period = fact.get_display_period_key() print(period) # "Q1 2024"` **Returns:** Period key like "Q1 2024", "FY 2023" EntityFactsParser Class ----------------------- Parser for converting SEC JSON data to enhanced EntityFacts format. ### Static Methods #### `parse_company_facts(facts_json: Dict[str, Any]) -> EntityFacts` Parse SEC company facts JSON to EntityFacts object. `from edgar.entity.parser import EntityFactsParser # Download SEC JSON facts_json = download_json(f"https://data.sec.gov/api/xbrl/companyfacts/CIK{cik:010d}.json") # Parse to enhanced format entity_facts = EntityFactsParser.parse_company_facts(facts_json)` **Parameters:** - `facts_json` (Dict): SEC company facts JSON data **Returns:** EntityFacts object Data Models ----------- ### DataQuality Enum Quality indicators for financial facts. `from edgar.entity.models import DataQuality DataQuality.HIGH # Direct from XBRL, validated DataQuality.MEDIUM # Derived or calculated DataQuality.LOW # Estimated or inferred` ### ConceptMetadata Class Metadata about financial concepts. `@dataclass class ConceptMetadata: concept: str label: str definition: str parent_concepts: List[str] child_concepts: List[str] # ... additional fields` Error Handling -------------- ### NoCompanyFactsFound Exception Raised when company facts cannot be found. `from edgar.entity.core import NoCompanyFactsFound try: facts = get_company_facts(invalid_cik) except NoCompanyFactsFound as e: print(f"No facts found: {e.message}")` Type Hints ---------- The API uses comprehensive type hints for better IDE support: `from typing import Optional, List, Dict, Any, Union from datetime import date from edgar.entity.entity_facts import EntityFacts from edgar.entity.models import FinancialFact from edgar.entity.query import FactQuery from edgar.entity.statement import FinancialStatement` Usage Patterns -------------- ### Method Chaining All query methods return the query object for chaining: `results = facts.query()\ .by_concept('Revenue')\ .by_fiscal_year(2024)\ .by_form_type('10-K')\ .sort_by('filing_date')\ .execute()` ### Error Handling The API uses graceful error handling: `# Methods return None instead of raising exceptions stmt = company.income_statement() # Returns None if no data if stmt: # Process statement pass` ### Performance Considerations * Use specific filters for better performance * Leverage caching by reusing EntityFacts objects * Use `count()` for existence checks before loading data * Prefer `latest()` over `execute()` when you need recent data only * * * _This API reference documents EdgarTools EntityFacts system. For usage examples and tutorials, see the [Company Facts Guide](https://edgartools.readthedocs.io/en/latest/guides/company-facts/) ._ Back to top --- # Cloud Storage (S3) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/cloud-storage/#cloud-storage-integration-guide) Cloud Storage Integration Guide =============================== This guide covers integrating EdgarTools with cloud storage providers (AWS S3, Google Cloud Storage, Azure Blob Storage) and S3-compatible services (Cloudflare R2, MinIO, DigitalOcean Spaces). Why Cloud Storage? ------------------ Cloud storage provides several advantages over local storage: | Benefit | Description | | --- | --- | | **Scalability** | Store terabytes of SEC data without local disk constraints | | **Team Sharing** | Multiple users/services access the same dataset | | **Durability** | Cloud providers offer 99.999999999% durability | | **Cost Efficiency** | Pay only for storage used; cheaper than provisioning servers | | **Global Access** | Access data from anywhere, any environment | Integration Approaches ---------------------- EdgarTools supports cloud storage through three mechanisms: 1. **`use_cloud_storage()`** - Native cloud integration via fsspec for **reading and writing** (recommended) 2. **`EDGAR_DATA_URL`** - Point to any HTTP endpoint for **reading** data 3. **`EDGAR_LOCAL_DATA_DIR` + FUSE** - Mount cloud storage as a local path (legacy) ### Approach Comparison | Feature | Native (`use_cloud_storage`) | EDGAR\_DATA\_URL | FUSE Mount | | --- | --- | --- | --- | | **Setup Complexity** | Simple | Simple | Complex | | **Read Data** | Yes | Yes | Yes | | **Write Data** | Yes | No | Yes | | **Requires Mount** | No | No | Yes | | **Platform Support** | All | All | Linux/macOS | | **Best For** | Full cloud integration | Read-only HTTP | Legacy systems | * * * Approach 1: EDGAR\_DATA\_URL (Read-Only) ---------------------------------------- The simplest approach for read-only access. Point EdgarTools to an HTTP endpoint serving your SEC data. ### How It Works `import os os.environ['EDGAR_DATA_URL'] = 'https://your-bucket.s3.amazonaws.com/edgar-data/' os.environ['EDGAR_USE_LOCAL_DATA'] = '1' from edgar import Company company = Company("AAPL") # Fetches from your S3 bucket` ### Setting Up S3 Static Website Hosting #### Step 1: Create and Configure S3 Bucket `# Create bucket aws s3 mb s3://my-edgar-data --region us-east-1 # Enable static website hosting aws s3 website s3://my-edgar-data \ --index-document index.html \ --error-document error.html` #### Step 2: Set Bucket Policy for Public Read Create `bucket-policy.json`: `{ "Version": "2012-10-17", "Statement": [ { "Sid": "PublicReadGetObject", "Effect": "Allow", "Principal": "*", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::my-edgar-data/*" } ] }` Apply the policy: `aws s3api put-bucket-policy \ --bucket my-edgar-data \ --policy file://bucket-policy.json` #### Step 3: Upload Your Data `# Sync local edgar data to S3 aws s3 sync ~/.edgar s3://my-edgar-data/ --storage-class STANDARD_IA` #### Step 4: Configure EdgarTools `import os # S3 static website URL format os.environ['EDGAR_DATA_URL'] = 'http://my-edgar-data.s3-website-us-east-1.amazonaws.com/' os.environ['EDGAR_USE_LOCAL_DATA'] = '1' from edgar import Company company = Company("AAPL") # Now reads from S3` ### Google Cloud Storage Setup `# Create bucket with uniform access gsutil mb -l us-central1 gs://my-edgar-data # Make bucket publicly readable gsutil iam ch allUsers:objectViewer gs://my-edgar-data # Upload data gsutil -m rsync -r ~/.edgar gs://my-edgar-data/` Configure EdgarTools: `import os os.environ['EDGAR_DATA_URL'] = 'https://storage.googleapis.com/my-edgar-data/' os.environ['EDGAR_USE_LOCAL_DATA'] = '1'` ### Azure Blob Storage Setup `# Create storage account and container az storage account create --name myedgardata --resource-group mygroup az storage container create --name edgar --account-name myedgardata --public-access blob # Upload data az storage blob upload-batch \ --account-name myedgardata \ --destination edgar \ --source ~/.edgar` Configure EdgarTools: `import os os.environ['EDGAR_DATA_URL'] = 'https://myedgardata.blob.core.windows.net/edgar/' os.environ['EDGAR_USE_LOCAL_DATA'] = '1'` ### Adding CloudFront CDN (Recommended for Production) For better performance and reduced S3 costs, add CloudFront: `# Create CloudFront distribution pointing to S3 aws cloudfront create-distribution \ --origin-domain-name my-edgar-data.s3.amazonaws.com \ --default-root-object index.html` Then use your CloudFront URL: `os.environ['EDGAR_DATA_URL'] = 'https://d1234567890.cloudfront.net/'` * * * Approach 2: FUSE Mount (Read/Write) ----------------------------------- For full read/write access, mount cloud storage as a local filesystem using FUSE (Filesystem in Userspace). ### FUSE Tool Comparison | Tool | Provider | Performance | Caching | Notes | | --- | --- | --- | --- | --- | | **s3fs-fuse** | AWS S3 | Moderate | Basic | Most compatible | | **goofys** | AWS S3 | Fast | Aggressive | Performance-focused | | **rclone mount** | All providers | Good | Configurable | Most versatile | | **gcsfuse** | Google Cloud | Good | Metadata | Official GCS tool | | **blobfuse2** | Azure | Good | File cache | Official Azure tool | ### s3fs-fuse Setup (AWS S3) #### Installation `# Ubuntu/Debian sudo apt-get install s3fs # macOS brew install s3fs # From source git clone https://github.com/s3fs-fuse/s3fs-fuse.git cd s3fs-fuse && ./autogen.sh && ./configure && make && sudo make install` #### Configuration Create credentials file: `echo "ACCESS_KEY_ID:SECRET_ACCESS_KEY" > ~/.passwd-s3fs chmod 600 ~/.passwd-s3fs` #### Mount the Bucket `# Create mount point mkdir -p /mnt/edgar-data # Mount with caching for better performance s3fs my-edgar-bucket /mnt/edgar-data \ -o passwd_file=~/.passwd-s3fs \ -o url=https://s3.amazonaws.com \ -o use_cache=/tmp/s3fs-cache \ -o ensure_diskfree=1024 \ -o parallel_count=15` #### Configure EdgarTools `import os os.environ['EDGAR_LOCAL_DATA_DIR'] = '/mnt/edgar-data' os.environ['EDGAR_USE_LOCAL_DATA'] = '1' from edgar import download_filings download_filings("2025-01-15") # Writes directly to S3!` ### goofys Setup (High Performance S3) goofys offers better performance than s3fs at the cost of some POSIX compliance. #### Installation `# Download binary wget https://github.com/kahing/goofys/releases/latest/download/goofys chmod +x goofys sudo mv goofys /usr/local/bin/` #### Mount `# Uses standard AWS credentials (~/.aws/credentials) goofys my-edgar-bucket /mnt/edgar-data # With specific profile goofys --profile production my-edgar-bucket /mnt/edgar-data # With caching goofys --stat-cache-ttl 1h --type-cache-ttl 1h my-edgar-bucket /mnt/edgar-data` ### rclone mount (Multi-Provider) rclone supports 40+ cloud storage providers with a unified interface. #### Installation `# Linux curl https://rclone.org/install.sh | sudo bash # macOS brew install rclone` #### Configure Provider `# Interactive configuration rclone config # Example: Configure S3 # Name: edgar-s3 # Type: s3 # Provider: AWS # Access key: (your key) # Secret key: (your secret) # Region: us-east-1` #### Mount `# Basic mount rclone mount edgar-s3:my-edgar-bucket /mnt/edgar-data # With VFS caching (recommended) rclone mount edgar-s3:my-edgar-bucket /mnt/edgar-data \ --vfs-cache-mode full \ --vfs-cache-max-age 24h \ --vfs-read-ahead 128M \ --buffer-size 128M \ --daemon` ### gcsfuse Setup (Google Cloud) `# Installation export GCSFUSE_REPO=gcsfuse-$(lsb_release -c -s) echo "deb https://packages.cloud.google.com/apt $GCSFUSE_REPO main" | sudo tee /etc/apt/sources.list.d/gcsfuse.list curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - sudo apt-get update && sudo apt-get install gcsfuse # Mount gcsfuse --implicit-dirs my-edgar-bucket /mnt/edgar-data` ### blobfuse2 Setup (Azure) `# Installation wget https://packages.microsoft.com/config/ubuntu/22.04/packages-microsoft-prod.deb sudo dpkg -i packages-microsoft-prod.deb sudo apt-get update && sudo apt-get install blobfuse2 # Create config file cat > ~/blobfuse2.yaml << EOF allow-other: true logging: type: syslog level: log_warning components: - libfuse - file_cache - attr_cache - azstorage libfuse: attribute-expiration-sec: 120 entry-expiration-sec: 120 file_cache: path: /tmp/blobfuse2 timeout-sec: 120 max-size-mb: 4096 azstorage: type: block account-name: myedgardata account-key: YOUR_ACCOUNT_KEY container: edgar EOF # Mount blobfuse2 mount /mnt/edgar-data --config-file=~/blobfuse2.yaml` ### Systemd Service (Auto-Mount on Boot) Create `/etc/systemd/system/edgar-s3.service`: `[Unit] Description=Mount S3 Edgar Data After=network-online.target [Service] Type=forking User=edgar ExecStart=/usr/local/bin/goofys -o allow_other my-edgar-bucket /mnt/edgar-data ExecStop=/bin/fusermount -u /mnt/edgar-data Restart=on-failure [Install] WantedBy=multi-user.target` Enable: `sudo systemctl enable edgar-s3 sudo systemctl start edgar-s3` * * * S3-Compatible Services ---------------------- ### Cloudflare R2 R2 offers S3-compatible storage with zero egress fees. #### Key Configuration R2 requires `region_name='auto'`: `# s3fs with R2 echo "R2_ACCESS_KEY:R2_SECRET_KEY" > ~/.passwd-r2 s3fs my-bucket /mnt/edgar-data \ -o passwd_file=~/.passwd-r2 \ -o url=https://ACCOUNT_ID.r2.cloudflarestorage.com \ -o use_path_request_style` #### rclone Configuration for R2 `rclone config # Name: edgar-r2 # Type: s3 # Provider: Cloudflare # access_key_id: (R2 access key) # secret_access_key: (R2 secret key) # endpoint: https://ACCOUNT_ID.r2.cloudflarestorage.com # acl: private` Mount: `rclone mount edgar-r2:my-edgar-bucket /mnt/edgar-data \ --vfs-cache-mode full` #### EDGAR\_DATA\_URL with R2 For read-only access via R2's public URL: `import os # Enable public access on your R2 bucket first os.environ['EDGAR_DATA_URL'] = 'https://pub-xxxxx.r2.dev/' os.environ['EDGAR_USE_LOCAL_DATA'] = '1'` ### MinIO MinIO is perfect for on-premises or private cloud deployments. `# s3fs with MinIO s3fs my-bucket /mnt/edgar-data \ -o passwd_file=~/.passwd-minio \ -o url=https://minio.example.com \ -o use_path_request_style # rclone config # Provider: Minio # Endpoint: https://minio.example.com` ### DigitalOcean Spaces `# rclone config # Provider: DigitalOcean # Endpoint: nyc3.digitaloceanspaces.com` * * * Hybrid Architecture Pattern --------------------------- Combine the best of both approaches for optimal performance: `┌─────────────────────────────────────────────────────────────┐ │ Hybrid Architecture │ ├─────────────────────────────────────────────────────────────┤ │ │ │ WRITES (Download/Sync) READS (Analysis) │ │ ┌─────────────────┐ ┌─────────────────┐ │ │ │ FUSE Mount │ │ EDGAR_DATA_URL │ │ │ │ (s3fs/rclone) │ │ + CloudFront │ │ │ └────────┬────────┘ └────────┬────────┘ │ │ │ │ │ │ ▼ ▼ │ │ ┌──────────────────────────────────────────────────┐ │ │ │ S3 Bucket (Origin) │ │ │ │ my-edgar-data │ │ │ └──────────────────────────────────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────┘` ### Implementation **Download Server (writes to S3):** `# Mount S3 for writing goofys my-edgar-bucket /mnt/edgar-data # Configure EdgarTools export EDGAR_LOCAL_DATA_DIR=/mnt/edgar-data export EDGAR_USE_LOCAL_DATA=1` `from edgar import download_filings download_filings("2025-01-01:2025-01-31") # Writes to S3` **Analysis Clients (reads via HTTP):** `import os # Fast reads via CloudFront os.environ['EDGAR_DATA_URL'] = 'https://d1234567890.cloudfront.net/' os.environ['EDGAR_USE_LOCAL_DATA'] = '1' from edgar import Company, get_filings # All reads go through CloudFront CDN filings = get_filings(form="10-K", year=2024)` * * * Sync Strategies --------------- ### Initial Bulk Upload `# Parallel upload with rclone rclone copy ~/.edgar edgar-s3:my-edgar-bucket \ --transfers 32 \ --checkers 16 \ --progress # Or with AWS CLI aws s3 sync ~/.edgar s3://my-edgar-bucket \ --storage-class STANDARD_IA` ### Incremental Daily Sync Create a cron job for daily updates: `# /etc/cron.d/edgar-sync 0 6 * * * edgar /usr/local/bin/edgar-daily-sync.sh` `edgar-daily-sync.sh`: `#!/bin/bash set -e # Download yesterday's filings locally first export EDGAR_LOCAL_DATA_DIR=/tmp/edgar-staging python -c " from edgar import download_filings from datetime import datetime, timedelta yesterday = (datetime.now() - timedelta(days=1)).strftime('%Y-%m-%d') download_filings(yesterday) " # Sync to S3 rclone sync /tmp/edgar-staging/filings edgar-s3:my-edgar-bucket/filings \ --transfers 16 \ --progress # Cleanup rm -rf /tmp/edgar-staging` ### Bidirectional Sync For teams with multiple download nodes: `# Use rclone bisync for two-way sync rclone bisync /mnt/local-edgar edgar-s3:my-edgar-bucket \ --resync \ --verbose` * * * Performance Optimization ------------------------ ### Caching Recommendations | Scenario | Tool | Cache Settings | | --- | --- | --- | | Frequent reads | goofys | `--stat-cache-ttl 1h` | | Large file writes | rclone | `--vfs-cache-mode full --vfs-cache-max-size 10G` | | Mixed workload | s3fs | `-o use_cache=/tmp/s3cache -o ensure_diskfree=2048` | ### Compression Filings are already compressed by EdgarTools. Additional S3 compression isn't necessary. ### Lifecycle Policies Reduce storage costs with lifecycle rules: `{ "Rules": [ { "ID": "MoveToIA", "Status": "Enabled", "Filter": {"Prefix": "filings/"}, "Transitions": [ { "Days": 30, "StorageClass": "STANDARD_IA" }, { "Days": 180, "StorageClass": "GLACIER" } ] } ] }` * * * Troubleshooting --------------- ### Common Issues **"Transport endpoint not connected"** `# FUSE mount crashed - remount sudo fusermount -u /mnt/edgar-data goofys my-edgar-bucket /mnt/edgar-data` **Slow performance with s3fs** `# Enable parallel requests and caching s3fs bucket /mnt/data \ -o parallel_count=20 \ -o multipart_size=52 \ -o use_cache=/tmp/s3cache \ -o max_stat_cache_size=100000` **Permission denied on mount** `# Add user_allow_other to /etc/fuse.conf echo "user_allow_other" | sudo tee -a /etc/fuse.conf # Mount with allow_other s3fs bucket /mnt/data -o allow_other` **R2 connection issues** `# Ensure region is set to 'auto' s3fs bucket /mnt/data \ -o url=https://ACCOUNT_ID.r2.cloudflarestorage.com \ -o use_path_request_style \ -o sigv2` ### Debugging `# s3fs debug mode s3fs bucket /mnt/data -d -f -o dbglevel=info # rclone debug rclone mount remote:bucket /mnt/data -vv --log-file=/tmp/rclone.log # Check mount status mount | grep fuse df -h /mnt/edgar-data` * * * Security Best Practices ----------------------- ### IAM Policies (AWS) Least-privilege policy for EdgarTools: `{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:GetObject", "s3:PutObject", "s3:ListBucket" ], "Resource": [ "arn:aws:s3:::my-edgar-bucket", "arn:aws:s3:::my-edgar-bucket/*" ] } ] }` ### Encryption `# Enable server-side encryption aws s3api put-bucket-encryption \ --bucket my-edgar-bucket \ --server-side-encryption-configuration \ '{"Rules":[{"ApplyServerSideEncryptionByDefault":{"SSEAlgorithm":"AES256"}}]}'` ### Private Access (No Public URLs) For internal-only access, skip the static website hosting and use: 1. FUSE mount with IAM credentials 2. VPC endpoints for AWS 3. Private connectivity for GCP/Azure * * * Native Cloud Support -------------------- EdgarTools provides native cloud storage support via `fsspec`, enabling seamless integration with S3, Google Cloud Storage, Azure Blob Storage, and S3-compatible services. ### Installation Install the cloud storage dependencies for your provider: `# AWS S3, Cloudflare R2, MinIO, DigitalOcean Spaces pip install "edgartools[s3]" # Google Cloud Storage pip install "edgartools[gcs]" # Azure Blob Storage pip install "edgartools[azure]" # All cloud providers pip install "edgartools[all-cloud]"` ### Basic Usage `import edgar # AWS S3 (uses default credentials from ~/.aws or environment) edgar.use_cloud_storage('s3://my-edgar-bucket/') # Now all operations use cloud storage company = edgar.Company("AAPL") filings = company.get_filings(form="10-K")` ### Provider Examples #### AWS S3 `import edgar # Using default AWS credentials edgar.use_cloud_storage('s3://my-edgar-bucket/') # With explicit credentials edgar.use_cloud_storage( 's3://my-edgar-bucket/', client_kwargs={ 'aws_access_key_id': 'YOUR_ACCESS_KEY', 'aws_secret_access_key': 'YOUR_SECRET_KEY', 'region_name': 'us-east-1' } )` #### Cloudflare R2 `import edgar edgar.use_cloud_storage( 's3://my-bucket/', client_kwargs={ 'endpoint_url': 'https://ACCOUNT_ID.r2.cloudflarestorage.com', 'region_name': 'auto' } )` #### Google Cloud Storage `import edgar # Using default GCP credentials edgar.use_cloud_storage('gs://my-edgar-bucket/') # With explicit project edgar.use_cloud_storage( 'gs://my-edgar-bucket/', client_kwargs={'project': 'my-project'} )` #### Azure Blob Storage `import edgar edgar.use_cloud_storage( 'az://my-container/edgar/', client_kwargs={ 'account_name': 'myaccount', 'account_key': 'YOUR_ACCOUNT_KEY' } )` #### MinIO (Self-Hosted S3) `import edgar edgar.use_cloud_storage( 's3://edgar-data/', client_kwargs={ 'endpoint_url': 'http://localhost:9000', 'aws_access_key_id': 'minioadmin', 'aws_secret_access_key': 'minioadmin' } )` ### Connection Verification By default, `use_cloud_storage()` verifies the connection by listing the bucket. This catches configuration errors early: `import edgar # Fails immediately if credentials are wrong or bucket doesn't exist edgar.use_cloud_storage('s3://my-bucket/') # Skip verification for faster startup (not recommended) edgar.use_cloud_storage('s3://my-bucket/', verify=False)` ### Disabling Cloud Storage `import edgar # Revert to local storage edgar.use_cloud_storage(disable=True)` ### Uploading Data to Cloud Storage EdgarTools provides two ways to populate your cloud storage with SEC data: #### Option 1: Download and Upload in One Step Use the `upload_to_cloud` parameter with `download_filings()`: `import edgar # Configure cloud storage first edgar.use_cloud_storage('s3://my-edgar-bucket/') # Download filings and upload to cloud automatically edgar.download_filings('2025-01-15', upload_to_cloud=True) # Download a date range edgar.download_filings('2025-01-01:2025-01-15', upload_to_cloud=True)` #### Option 2: Sync Existing Local Data Use `sync_to_cloud()` to upload data you've already downloaded locally: `import edgar # Configure cloud storage edgar.use_cloud_storage('s3://my-edgar-bucket/') # Sync all local filings to cloud result = edgar.sync_to_cloud('filings') print(f"Uploaded: {result['uploaded']}, Skipped: {result['skipped']}") # Sync specific date directory edgar.sync_to_cloud('filings/20250115') # Preview what would be uploaded (dry run) edgar.sync_to_cloud('filings', dry_run=True) # Overwrite existing files in cloud edgar.sync_to_cloud('filings', overwrite=True)` #### sync\_to\_cloud() Parameters | Parameter | Type | Default | Description | | --- | --- | --- | --- | | `source_path` | str | None | Subdirectory to sync (e.g., 'filings', 'filings/20250115') | | `pattern` | str | '\*_/_' | Glob pattern for files to sync | | `batch_size` | int | 20 | Number of concurrent uploads | | `overwrite` | bool | False | Overwrite existing files in cloud | | `dry_run` | bool | False | Preview without uploading | #### Return Value `sync_to_cloud()` returns a dict with upload statistics: `{ 'uploaded': 150, # Files successfully uploaded 'skipped': 50, # Files already in cloud (when overwrite=False) 'failed': 0, # Files that failed to upload 'errors': [] # Error messages for failed uploads }` ### Features | Feature | Description | | --- | --- | | **Cross-platform** | Works on Windows, macOS, and Linux | | **No FUSE required** | Pure Python implementation | | **Transparent compression** | Handles `.gz` files automatically | | **Full read/write** | Both reading and writing supported | | **Provider agnostic** | Same API for all cloud providers | * * * Summary ------- | Use Case | Recommended Approach | | --- | --- | | **Native cloud support** | `use_cloud_storage()` (recommended) | | **Read-only HTTP access** | `EDGAR_DATA_URL` + static website | | **Legacy FUSE mount** | goofys or rclone mount | | **On-premises** | MinIO + `use_cloud_storage()` | | **Zero egress costs** | Cloudflare R2 | ### Quick Start **Native cloud storage (recommended):** `import edgar # Install: pip install "edgartools[s3]" edgar.use_cloud_storage('s3://my-edgar-bucket/') # Read from cloud company = edgar.Company("AAPL") # Write to cloud edgar.download_filings('2025-01-15', upload_to_cloud=True) # Or sync existing local data edgar.sync_to_cloud('filings')` **Read-only via HTTP:** `import os os.environ['EDGAR_DATA_URL'] = 'https://your-bucket.s3.amazonaws.com/' os.environ['EDGAR_USE_LOCAL_DATA'] = '1'` **Legacy FUSE mount (Linux/macOS):** `goofys my-edgar-bucket /mnt/edgar-data export EDGAR_LOCAL_DATA_DIR=/mnt/edgar-data export EDGAR_USE_LOCAL_DATA=1` For questions or feedback, see [Discussion #507](https://github.com/dgunning/edgartools/discussions/507) . Back to top --- # Entity Facts - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/api/entity-facts-reference/#entityfacts-api-reference) EntityFacts API Reference ========================= Complete API documentation for the enhanced EntityFacts system, including all classes, methods, and data models. Overview -------- The EntityFacts API provides structured access to SEC company financial data with AI-ready features, powerful querying capabilities, and professional formatting. The system consists of several key components: * **EntityFacts** - Main class for accessing company facts * **FactQuery** - Fluent query builder for advanced filtering * **FinancialStatement** - Formatted display wrapper for financial data * **FinancialFact** - Individual fact data model with rich metadata EntityFacts Class ----------------- The main entry point for accessing company financial facts. ### Constructor `EntityFacts(cik: int, name: str, facts: List[FinancialFact])` **Parameters:** - `cik` (int): Company CIK number - `name` (str): Company name \- `facts` (List\[FinancialFact\]): List of financial facts ### Properties #### Core Properties #### `cik: int` The company's CIK (Central Index Key) number. `facts = company.facts print(facts.cik) # 320193` #### `name: str` The company's official name. `facts = company.facts print(facts.name) # "Apple Inc."` #### DEI Properties #### `shares_outstanding: Optional[float]` Number of common shares outstanding. `shares = facts.shares_outstanding if shares: print(f"Shares Outstanding: {shares:,.0f}")` #### `public_float: Optional[float]` Public float value in dollars. `float_val = facts.public_float if float_val: print(f"Public Float: ${float_val:,.0f}")` #### `shares_outstanding_fact: Optional[FinancialFact]` Full fact object for shares outstanding with metadata. `fact = facts.shares_outstanding_fact if fact: print(f"Shares: {fact.get_formatted_value()} as of {fact.period_end}")` #### `public_float_fact: Optional[FinancialFact]` Full fact object for public float with metadata. `fact = facts.public_float_fact if fact: print(f"Float: {fact.get_formatted_value()} as of {fact.period_end}")` ### Core Methods #### Query Interface #### `query() -> FactQuery` Start building a facts query using the fluent interface. `query = facts.query() results = query.by_concept('Revenue').latest(4)` **Returns:** FactQuery builder instance #### `get_fact(concept: str, period: Optional[str] = None) -> Optional[FinancialFact]` Get a single fact by concept name. `# Use the full XBRL concept name or the lowercase label revenue_fact = facts.get_fact('us-gaap:Revenues') q1_revenue = facts.get_fact('us-gaap:Revenues', '2024-Q1') # Lowercase labels also work revenue_fact = facts.get_fact('revenues')` **Parameters:** - `concept` (str): XBRL concept name (e.g. `'us-gaap:Revenues'`) or lowercase label (e.g. `'revenues'`). Use `search_concepts()` to find valid names. - `period` (str, optional): Period in format "YYYY-QN" or "YYYY-FY" **Returns:** Most recent matching fact or None `time_series(concept: str, periods: int = 20) -> pd.DataFrame` Get time series data for a concept. `revenue_ts = facts.time_series('Revenue', periods=8)` **Parameters:** - `concept` (str): Concept name or label - `periods` (int): Number of periods to retrieve (default: 20) **Returns:** DataFrame with time series data ### Financial Statement Methods `income_statement(periods: int = 4, period_length: Optional[int] = None, as_dataframe: bool = False, annual: bool = True)` Get income statement facts formatted as a financial statement. `# Default: 4 annual periods, formatted display stmt = facts.income_statement() # 8 quarterly periods as DataFrame df = facts.income_statement(periods=8, annual=False, as_dataframe=True)` **Parameters:** - `periods` (int): Number of periods to retrieve (default: 4) - `period_length` (int, optional): Filter by period length in months (3=quarterly, 12=annual) - `as_dataframe` (bool): If True, return DataFrame; if False, return FinancialStatement (default: False) - `annual` (bool): If True, prefer annual periods; if False, prefer quarterly (default: True) **Returns:** FinancialStatement or DataFrame #### `balance_sheet(periods: int = 4, as_of: Optional[date] = None, as_dataframe: bool = False, annual: bool = True)` Get balance sheet facts for periods or point-in-time. `# Multi-period balance sheet stmt = facts.balance_sheet(periods=4) # Point-in-time snapshot snapshot = facts.balance_sheet(as_of=date(2024, 12, 31))` **Parameters:** - `periods` (int): Number of periods to retrieve (default: 4) - `as_of` (date, optional): Get snapshot as of specific date - `as_dataframe` (bool): If True, return DataFrame; if False, return FinancialStatement (default: False) - `annual` (bool): If True, prefer annual periods (default: True) **Returns:** FinancialStatement or DataFrame #### `cash_flow(periods: int = 4, period_length: Optional[int] = None, as_dataframe: bool = False, annual: bool = True)` Get cash flow statement facts. `# Annual cash flow trends stmt = facts.cashflow_statement(periods=5, annual=True)` **Parameters:** - `periods` (int): Number of periods to retrieve (default: 4) - `period_length` (int, optional): Filter by period length in months - `as_dataframe` (bool): If True, return DataFrame; if False, return FinancialStatement (default: False) - `annual` (bool): If True, prefer annual periods (default: True) **Returns:** FinancialStatement or DataFrame ### DEI Methods #### `dei_facts(as_of: Optional[date] = None) -> pd.DataFrame` Get Document and Entity Information facts. `# Latest DEI facts dei = facts.dei_facts() # DEI facts as of specific date dei = facts.dei_facts(as_of=date(2024, 12, 31))` **Parameters:** - `as_of` (date, optional): Get facts as of specific date **Returns:** DataFrame with DEI facts #### `entity_info() -> Dict[str, Any]` Get key entity information as a clean dictionary. `info = facts.entity_info() print(info['entity_name']) print(info['shares_outstanding'])` **Returns:** Dictionary with entity information ### AI/LLM Methods #### `to_llm_context(focus_areas: Optional[List[str]] = None, time_period: str = "recent") -> Dict[str, Any]` Generate comprehensive context for LLM analysis. `context = facts.to_llm_context( focus_areas=['profitability', 'growth'], time_period='5Y' )` **Parameters:** - `focus_areas` (List\[str\], optional): Areas to emphasize (\['profitability', 'growth', 'liquidity'\]) - `time_period` (str): Time period to analyze ('recent', '5Y', '10Y', 'all') (default: 'recent') **Returns:** Dictionary with structured LLM context #### `to_agent_tools() -> List[Dict[str, Any]]` Export facts as MCP-compatible tools for AI agents. `tools = facts.to_agent_tools()` **Returns:** List of tool definitions ### Magic Methods #### `__len__() -> int` Get total number of facts. `total_facts = len(facts)` #### `__iter__() -> Iterator[FinancialFact]` Iterate over all facts. `for fact in facts: print(f"{fact.concept}: {fact.numeric_value}")` FactQuery Class --------------- Fluent query builder for advanced fact filtering and analysis. ### Constructor Created via `EntityFacts.query()` method. Do not instantiate directly. ### Filtering Methods #### Concept Filtering #### `by_concept(concept: str, exact: bool = False) -> FactQuery` Filter by concept name or pattern. `# Fuzzy matching (default) revenue_facts = query.by_concept('Revenue') # Exact matching exact_revenue = query.by_concept('us-gaap:Revenue', exact=True)` **Parameters:** - `concept` (str): Concept name or label to match - `exact` (bool): If True, require exact match (default: False) #### `by_label(label: str, fuzzy: bool = True) -> FactQuery` Filter by human-readable label. `# Fuzzy label matching facts = query.by_label('Total Revenue', fuzzy=True) # Exact label matching facts = query.by_label('Revenue', fuzzy=False)` **Parameters:** - `label` (str): Label to match - `fuzzy` (bool): Use fuzzy matching (default: True) #### Time-Based Filtering #### `by_fiscal_year(year: int) -> FactQuery` Filter by fiscal year. `fy2024_facts = query.by_fiscal_year(2024)` **Parameters:** - `year` (int): Fiscal year to filter by #### `by_fiscal_period(period: str) -> FactQuery` Filter by fiscal period. `q1_facts = query.by_fiscal_period('Q1') fy_facts = query.by_fiscal_period('FY')` **Parameters:** - `period` (str): Fiscal period ('FY', 'Q1', 'Q2', 'Q3', 'Q4') #### `by_period_length(months: int) -> FactQuery` Filter by period length in months. `# Quarterly periods (3 months) quarterly = query.by_period_length(3) # Annual periods (12 months) annual = query.by_period_length(12)` **Parameters:** - `months` (int): Period length (3=quarterly, 12=annual, 9=YTD) #### `date_range(start: date, end: date) -> FactQuery` Filter by date range. `recent_facts = query.date_range( start=date(2023, 1, 1), end=date(2024, 12, 31) )` **Parameters:** - `start` (date): Start date (inclusive) - `end` (date): End date (inclusive) #### `as_of(as_of_date: date) -> FactQuery` Get facts as of specific date (point-in-time). `snapshot = query.as_of(date(2024, 6, 30))` **Parameters:** - `as_of_date` (date): Date for point-in-time view #### Statement and Form Filtering #### `by_statement_type(statement_type: str) -> FactQuery` Filter by financial statement type. `income_facts = query.by_statement_type('IncomeStatement') balance_facts = query.by_statement_type('BalanceSheet') cash_facts = query.by_statement_type('CashFlow')` **Parameters:** - `statement_type` (str): Statement type ('IncomeStatement', 'BalanceSheet', 'CashFlow') #### `by_form_type(form_type: Union[str, List[str]]) -> FactQuery` Filter by SEC form type. `# Single form type annual_facts = query.by_form_type('10-K') # Multiple form types periodic_facts = query.by_form_type(['10-K', '10-Q'])` **Parameters:** - `form_type` (str or List\[str\]): Form type(s) to filter by #### Quality Filtering #### `high_quality_only() -> FactQuery` Filter to only high-quality, audited facts. `quality_facts = query.high_quality_only()` #### `min_confidence(threshold: float) -> FactQuery` Filter by minimum confidence score. `confident_facts = query.min_confidence(0.9)` **Parameters:** - `threshold` (float): Minimum confidence score (0.0 to 1.0) #### Special Queries #### `latest_instant() -> FactQuery` Filter to most recent instant facts (for balance sheet items). `latest_balance = query.by_statement_type('BalanceSheet').latest_instant()` #### `latest_periods(n: int = 4, annual: bool = True) -> FactQuery` Get facts from the n most recent periods. `# Latest 4 annual periods only recent = query.latest_periods(4, annual=True) # Latest 8 periods, any type recent = query.latest_periods(8, annual=False)` **Parameters:** - `n` (int): Number of recent periods (default: 4) - `annual` (bool): If True, only use annual periods; if False, use all period types (default: True) ### Sorting and Limiting #### `sort_by(field: str, ascending: bool = True) -> FactQuery` Sort results by field. `# Sort by filing date (newest first) sorted_facts = query.sort_by('filing_date', ascending=False) # Sort by fiscal year sorted_facts = query.sort_by('fiscal_year')` **Parameters:** - `field` (str): Field name to sort by - `ascending` (bool): Sort order (default: True) #### `latest(n: int = 1) -> List[FinancialFact]` Get the n most recent facts. `latest_revenue = query.by_concept('Revenue').latest(5)` **Parameters:** - `n` (int): Number of facts to return (default: 1) **Returns:** List of facts (executes query immediately) ### Execution Methods #### `execute() -> List[FinancialFact]` Execute query and return matching facts. `facts = query.by_concept('Revenue').by_fiscal_year(2024).execute()` **Returns:** List of FinancialFact objects #### `count() -> int` Get count of facts matching current filters. `revenue_count = query.by_concept('Revenue').count()` **Returns:** Number of matching facts ### Output Methods #### `to_dataframe(*columns) -> pd.DataFrame` Convert results to pandas DataFrame. `# All columns df = query.by_concept('Revenue').to_dataframe() # Selected columns df = query.by_concept('Revenue').to_dataframe( 'label', 'numeric_value', 'fiscal_period' )` **Parameters:** - `*columns` (str): Optional column names to include **Returns:** DataFrame with query results #### `pivot_by_period(return_statement: bool = True) -> Union[FinancialStatement, pd.DataFrame]` Pivot facts to show concepts as rows and periods as columns. `# Formatted financial statement stmt = query.by_statement_type('IncomeStatement').pivot_by_period() # Raw DataFrame df = query.by_statement_type('IncomeStatement').pivot_by_period(return_statement=False)` **Parameters:** - `return_statement` (bool): If True, return FinancialStatement; if False, return DataFrame (default: True) **Returns:** FinancialStatement or DataFrame #### `to_llm_context() -> List[Dict[str, Any]]` Convert results to LLM-friendly context. `llm_data = query.by_concept('Revenue').to_llm_context()` **Returns:** List of fact contexts for LLM consumption FinancialStatement Class ------------------------ Wrapper around pandas DataFrame for financial statements with intelligent formatting. ### Constructor `FinancialStatement( data: pd.DataFrame, statement_type: str, entity_name: str = "", period_lengths: Optional[List[str]] = None, mixed_periods: bool = False )` **Parameters:** - `data` (pd.DataFrame): Financial data - `statement_type` (str): Statement type - `entity_name` (str): Company name - `period_lengths` (List\[str\], optional): Period lengths in data - `mixed_periods` (bool): Whether data contains mixed period lengths ### Properties #### `shape: tuple` Shape of the underlying DataFrame. `stmt = company.income_statement() print(stmt.shape) # (10, 4)` #### `columns: pd.Index` Column names of the statement. `periods = stmt.columns print(list(periods)) # ['FY 2024', 'FY 2023', 'FY 2022', 'FY 2021']` #### `index: pd.Index` Row labels (concept names). `concepts = stmt.index print(list(concepts)) # ['Revenue', 'Cost of Revenue', 'Gross Profit', ...]` #### `empty: bool` Whether the statement is empty. `if not stmt.empty: print("Statement has data")` ### Methods #### `to_numeric() -> pd.DataFrame` Get underlying numeric DataFrame for calculations. `stmt = company.income_statement() numeric_data = stmt.to_numeric() growth_rates = numeric_data.pct_change(axis=1)` **Returns:** DataFrame with original numeric values #### `get_concept(concept_name: str) -> Optional[pd.Series]` Get data for specific concept across all periods. `revenue_series = stmt.get_concept('Revenue') if revenue_series is not None: print(revenue_series)` **Parameters:** - `concept_name` (str): Name of concept to retrieve **Returns:** Series with values across periods, or None #### `calculate_growth(concept_name: str, periods: int = 2) -> Optional[pd.Series]` Calculate period-over-period growth for a concept. `revenue_growth = stmt.calculate_growth('Revenue', periods=1)` **Parameters:** - `concept_name` (str): Name of concept - `periods` (int): Number of periods for growth calculation (default: 2) **Returns:** Series with growth rates, or None #### `format_value(value: float, concept_label: str) -> str` Format a single value based on its concept. `formatted = stmt.format_value(1234567, 'Revenue') print(formatted) # "$1,234,567"` **Parameters:** - `value` (float): Numeric value to format - `concept_label` (str): Label of financial concept **Returns:** Formatted string #### `to_llm_context() -> Dict[str, Any]` Generate LLM-friendly context from the statement. `context = stmt.to_llm_context()` **Returns:** Dictionary with structured financial data ### Display Methods The FinancialStatement class provides rich display capabilities: * **Jupyter Notebooks**: Automatic HTML rendering with professional styling * **Console**: Formatted text output with proper alignment * **Rich Integration**: Compatible with Rich library for enhanced terminal display FinancialFact Class ------------------- Individual financial fact with rich metadata and AI-ready features. ### Constructor `FinancialFact( concept: str, taxonomy: str, label: str, value: Union[float, int, str], numeric_value: Optional[float], unit: str, scale: Optional[int] = None, # ... additional parameters )` ### Core Attributes #### `concept: str` Standardized concept identifier (e.g., 'us-gaap:Revenue'). #### `taxonomy: str` Taxonomy namespace (us-gaap, ifrs, etc.). #### `label: str` Human-readable label. #### `value: Union[float, int, str]` The actual fact value. #### `numeric_value: Optional[float]` Numeric representation for calculations. #### `unit: str` Unit of measure (USD, shares, etc.). #### `scale: Optional[int]` Scale factor (1000, 1000000, etc.). ### Temporal Attributes #### `period_start: Optional[date]` Period start date (for duration facts). #### `period_end: date` Period end date. #### `period_type: Literal['instant', 'duration']` Type of period. #### `fiscal_year: int` Fiscal year. #### `fiscal_period: str` Fiscal period (FY, Q1, Q2, Q3, Q4). ### Filing Context #### `filing_date: date` Date the fact was filed with SEC. #### `form_type: str` SEC form type (10-K, 10-Q, etc.). #### `accession: str` SEC accession number. ### Quality Indicators #### `data_quality: DataQuality` Data quality enum (HIGH, MEDIUM, LOW). #### `is_audited: bool` Whether the fact is from audited filing. #### `confidence_score: float` Confidence score (0.0 to 1.0). ### AI-Ready Attributes #### `semantic_tags: List[str]` Semantic tags for AI processing. #### `business_context: str` Business context description. ### Methods #### `to_llm_context() -> Dict[str, Any]` Generate rich context for LLM consumption. `fact = facts.get_fact('us-gaap:Revenues') context = fact.to_llm_context() print(context['concept']) print(context['value']) print(context['period'])` **Returns:** Dictionary with formatted context #### `get_formatted_value() -> str` Format the numeric value for display. `fact = facts.get_fact('us-gaap:Revenues') formatted = fact.get_formatted_value() print(formatted) # "365,817,000,000"` **Returns:** Formatted string representation #### `get_display_period_key() -> str` Generate display-friendly period key. `fact = facts.get_fact('us-gaap:Revenues') period = fact.get_display_period_key() print(period) # "Q1 2024"` **Returns:** Period key like "Q1 2024", "FY 2023" EntityFactsParser Class ----------------------- Parser for converting SEC JSON data to enhanced EntityFacts format. ### Static Methods #### `parse_company_facts(facts_json: Dict[str, Any]) -> EntityFacts` Parse SEC company facts JSON to EntityFacts object. `from edgar.entity.parser import EntityFactsParser # Download SEC JSON facts_json = download_json(f"https://data.sec.gov/api/xbrl/companyfacts/CIK{cik:010d}.json") # Parse to enhanced format entity_facts = EntityFactsParser.parse_company_facts(facts_json)` **Parameters:** - `facts_json` (Dict): SEC company facts JSON data **Returns:** EntityFacts object Data Models ----------- ### DataQuality Enum Quality indicators for financial facts. `from edgar.entity.models import DataQuality DataQuality.HIGH # Direct from XBRL, validated DataQuality.MEDIUM # Derived or calculated DataQuality.LOW # Estimated or inferred` ### ConceptMetadata Class Metadata about financial concepts. `@dataclass class ConceptMetadata: concept: str label: str definition: str parent_concepts: List[str] child_concepts: List[str] # ... additional fields` Error Handling -------------- ### NoCompanyFactsFound Exception Raised when company facts cannot be found. `from edgar.entity.core import NoCompanyFactsFound try: facts = get_company_facts(invalid_cik) except NoCompanyFactsFound as e: print(f"No facts found: {e.message}")` Type Hints ---------- The API uses comprehensive type hints for better IDE support: `from typing import Optional, List, Dict, Any, Union from datetime import date from edgar.entity.entity_facts import EntityFacts from edgar.entity.models import FinancialFact from edgar.entity.query import FactQuery from edgar.entity.statement import FinancialStatement` Usage Patterns -------------- ### Method Chaining All query methods return the query object for chaining: `results = facts.query()\ .by_concept('Revenue')\ .by_fiscal_year(2024)\ .by_form_type('10-K')\ .sort_by('filing_date')\ .execute()` ### Error Handling The API uses graceful error handling: `# Methods return None instead of raising exceptions stmt = company.income_statement() # Returns None if no data if stmt: # Process statement pass` ### Performance Considerations * Use specific filters for better performance * Leverage caching by reusing EntityFacts objects * Use `count()` for existence checks before loading data * Prefer `latest()` over `execute()` when you need recent data only * * * _This API reference documents EdgarTools EntityFacts system. For usage examples and tutorials, see the [Company Facts Guide](https://edgartools.readthedocs.io/en/stable/guides/company-facts/) ._ Back to top --- # Company Facts - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/company-facts/#company-facts-query-historical-sec-financial-data-with-python) Company Facts: Query Historical SEC Financial Data with Python ============================================================== The Company Facts API provides comprehensive access to SEC financial data through an intuitive, AI-ready interface. Get financial statements, key metrics, and detailed company information with just a few lines of code. ✨ **Latest Features:** * **Enhanced Value Formatting**: Full numbers with commas (1,000,000,000) by default, with optional concise format ($1.0B) * **Multi-Period Statements**: Rich hierarchical display showing multiple periods side-by-side * **LLM Integration**: Built-in `to_llm_context()` method for AI consumption * **Web Rendering Support**: Easy iteration over statement items with comprehensive web API methods * **Improved Visual Display**: Professional formatting with color-coded values and hierarchical structure Quick Start ----------- `from edgar import Company # Get any public company company = Company('AAPL') # Ticker symbol # or company = Company(320193) # CIK number # Access key metrics instantly print(f"Shares Outstanding: {company.shares_outstanding:,.0f}") print(f"Public Float: ${company.public_float:,.0f}") # Get enhanced multi-period financial statements income_stmt = company.income_statement() # Shows multiple periods with hierarchy balance_sheet = company.balance_sheet() cash_flow = company.cashflow_statement() print(income_stmt) # Rich multi-period display # Get concise format for quick overview income_compact = company.income_statement(concise_format=True) print(income_compact) # Shows $1.0B instead of $1,000,000,000` Key Features ------------ * **🚀 Zero Setup** - Works immediately with existing Company objects * **💰 Full Precision** - Full numbers with commas by default, optional concise formatting * **📊 Enhanced Display** - Multi-period hierarchical statements with rich formatting * **🛡️ Error Resilient** - Graceful handling of missing data with intelligent fallbacks * **🤖 AI-Ready** - Built-in LLM context generation with structured data output * **🌐 Web Integration** - Easy iteration methods and rendering support for web applications * **⚡ Performance Optimized** - Intelligent caching and efficient data structures * **🎨 Professional Formatting** - Color-coded values, hierarchical structure, and smart spacing Core Properties --------------- ### Company Metrics Access essential company information through simple properties: `company = Company('TSLA') # Key financial metrics print(f"Shares Outstanding: {company.shares_outstanding:,.0f}") print(f"Public Float: ${company.public_float:,.0f}") # Check if facts are available if company.facts: print(f"Total facts available: {len(company.facts):,}")` **Available Properties:** * `company.facts` - Access to the full EntityFacts object * `company.shares_outstanding` - Number of shares outstanding * `company.public_float` - Public float value in dollars Financial Statements -------------------- ### Income Statement Get hierarchical income statement data with flexible period options: `# Default: 4 annual periods, enhanced multi-period display income_stmt = company.income_statement() print(income_stmt) # Rich hierarchical display with multiple periods # Get 8 quarterly periods with full number formatting quarterly = company.income_statement(periods=8, annual=False) # Use concise format for quick analysis ($1.0B vs $1,000,000,000) compact = company.income_statement(concise_format=True) # Get raw DataFrame for analysis df = company.income_statement(periods=4, as_dataframe=True) # Convert to LLM-friendly format llm_data = income_stmt.to_llm_context() print(llm_data['key_metrics']) # Automatic ratio calculations` ### Balance Sheet Access hierarchical balance sheet data for point-in-time or trend analysis: `# Enhanced multi-period balance sheet with hierarchy balance_sheet = company.balance_sheet(periods=4) print(balance_sheet) # Shows Assets, Liabilities, Equity sections # Point-in-time snapshot as of specific date from datetime import date snapshot = company.balance_sheet(as_of=date(2024, 12, 31)) # Concise format for executive summaries exec_summary = company.balance_sheet(concise_format=True) # Raw data for calculations df = company.balance_sheet(periods=3, as_dataframe=True) # Web rendering support - iterate over items for item in balance_sheet: print(f"{item.label}: {item.get_display_value(balance_sheet.periods[0])}")` ### Cash Flow Statement Analyze hierarchical cash flow patterns across periods: `# Enhanced annual cash flow with operating/investing/financing sections cash_flow = company.cashflow_statement(periods=5, annual=True) print(cash_flow) # Rich display with cash flow categories # Quarterly cash flow analysis with full formatting quarterly_cf = company.cashflow_statement(periods=8, annual=False) # Executive dashboard format exec_cf = company.cashflow_statement(concise_format=True) # Generate analysis context for AI ai_context = cash_flow.to_llm_context(include_metadata=True) print(ai_context['key_metrics']) # Automatic cash flow metrics` Method Parameters ----------------- All financial statement methods support consistent parameters: | Parameter | Type | Default | Description | | --- | --- | --- | --- | | `periods` | int | 4 | Number of periods to retrieve | | `annual` | bool | True | If True, prefer annual periods; if False, get quarterly | | `as_dataframe` | bool | False | If True, return raw DataFrame; if False, return MultiPeriodStatement | | `concise_format` | bool | False | If True, display as $1.0B; if False, display as $1,000,000,000 | **Special Parameters:** - `balance_sheet()` also supports `as_of` parameter for point-in-time views Return Types ------------ ### MultiPeriodStatement Objects (Default) When `as_dataframe=False` (default), methods return enhanced `MultiPeriodStatement` objects with: * **Hierarchical Structure**: Organized sections with proper parent-child relationships * **Multi-Period Display**: Side-by-side period comparison with rich formatting * **Smart Value Formatting**: Full numbers ($1,000,000,000) by default, per-share amounts as decimals * **Color-Coded Display**: Green/red values, bold totals, hierarchical indentation * **Web Rendering Support**: Easy iteration and item access for web applications * **LLM Integration**: Built-in context generation for AI analysis `stmt = company.income_statement() # Rich multi-period display (automatic in notebooks) print(stmt) # Convert to DataFrame for analysis df = stmt.to_dataframe() revenue_growth = df.loc['Revenue'].pct_change() # Generate LLM-friendly context llm_data = stmt.to_llm_context() print(llm_data['key_metrics']['profit_margin_fy_2024']) # Iterate over items for web rendering for item in stmt.iter_with_values(): print(f"{item.label}: {item.get_display_value(stmt.periods[0])}") # Get specific item revenue_item = stmt.find_item('Revenue') if revenue_item: print(f"Revenue trend: {revenue_item.values}")` ### DataFrame Objects When `as_dataframe=True`, methods return pandas DataFrames with enhanced structure: `df = company.income_statement(as_dataframe=True) # Enhanced DataFrame with metadata columns print(df.columns) # Includes: periods, depth, is_total, section, confidence print(df.dtypes) print(df.describe()) # Access financial data revenue_series = df.loc['us-gaap:Revenues'] # Full concept names as index print(df[df['is_total']]) # Filter to total/subtotal rows only print(df[df['section'] == 'Revenue']) # Filter by statement section` Enhanced Features ----------------- ### Value Formatting Options The API now provides flexible value formatting to suit different use cases: `# Full precision formatting (default) - best for analysis stmt_full = company.income_statement(concise_format=False) print(stmt_full) # Shows: $391,035,000,000 # Concise formatting - best for presentations and dashboards stmt_concise = company.income_statement(concise_format=True) print(stmt_concise) # Shows: $391.0B # Per-share amounts are always displayed as decimals # Example: Earnings Per Share shows as "2.97" not "$2.97" or "$2,970,000,000"` **Formatting Rules:** * **Default (`concise_format=False`)**: Full numbers with commas ($1,000,000,000) * **Concise (`concise_format=True`)**: Scaled format ($1.0B, $500.3M) * **Per-Share Values**: Always decimal format (2.97) regardless of setting * **Negative Values**: Properly formatted with minus signs * **Zero/Null Values**: Displayed as "-" for clean presentation ### LLM Integration and AI Context Generate structured data optimized for AI and LLM consumption: `stmt = company.income_statement(periods=4) # Generate LLM-friendly context llm_context = stmt.to_llm_context( include_metadata=True, # Include data quality metrics include_hierarchy=False, # Flatten for simplicity (default) flatten_values=True # Create period-prefixed keys (default) ) print("LLM Context Structure:") print(f"Company: {llm_context['company']}") print(f"Statement Type: {llm_context['statement_type']}") print(f"Periods: {llm_context['periods']}") print(f"Data Quality: {llm_context['metadata']['quality_indicators']}") # Access flattened financial data financial_data = llm_context['data'] print(f"Revenue FY 2024: ${financial_data.get('revenue_fy_2024', 0):,.0f}") print(f"Revenue FY 2023: ${financial_data.get('revenue_fy_2023', 0):,.0f}") # Automatic ratio calculations key_metrics = llm_context.get('key_metrics', {}) if 'profit_margin_fy_2024' in key_metrics: print(f"Current Profit Margin: {key_metrics['profit_margin_fy_2024']:.1%}") # Feed to LLM for analysis import json analysis_prompt = f""" Analyze this financial data for {llm_context['company']}: {json.dumps(llm_context, indent=2)} Provide insights on profitability trends and growth patterns. """` ### Web Application Integration Easy iteration and rendering support for web applications: `stmt = company.income_statement(periods=4) # Basic iteration over all items for item in stmt: print(f"{item.label}: {item.get_display_value(stmt.periods[0])}") # Iterate with hierarchy information for item in stmt.iter_hierarchy(): indent = " " * item.depth parent_info = f" (parent: {item.parent.label})" if item.parent else "" print(f"{indent}{item.label}{parent_info}") # Only items with values (skip empty rows) for item in stmt.iter_with_values(): values_summary = ", ".join([ f"{period}: {item.get_display_value(period)}" for period in stmt.periods if item.values.get(period) ]) print(f"{item.label} -> {values_summary}") # Find specific items revenue_item = stmt.find_item('Revenue') if revenue_item: print(f"Found Revenue: {revenue_item.values}") # Convert to web-friendly format web_data = stmt.to_dict() # Nested dictionary flat_data = stmt.to_flat_list() # Flat list for tables # Period comparison analysis comparison = stmt.get_period_comparison() for concept, analysis in comparison.items(): if analysis['growth_rate']: print(f"{concept}: {analysis['growth_rate']:.1%} growth")` ### Advanced Statement Features #### Smart Hierarchical Organization Statements now display with intelligent hierarchy based on accounting standards: `stmt = company.income_statement() print(stmt) # Shows: # Revenue # Product Revenue # Service Revenue # Cost of Revenue # Cost of Product Sales # Cost of Services # Gross Profit [calculated] # Operating Expenses # Research and Development # Sales and Marketing # Operating Income [calculated]` #### Professional Visual Display * **Color Coding**: Green for positive values, red for negative * **Bold Formatting**: Totals and subtotals are emphasized * **Hierarchical Indentation**: Clear parent-child relationships * **Confidence Indicators**: Low-confidence items marked with ◦ * **Smart Spacing**: Separators after major sections #### Enhanced Data Quality Statements include data quality metadata: `stmt = company.income_statement() # Check overall statement quality if hasattr(stmt, 'canonical_coverage'): print(f"Canonical Coverage: {stmt.canonical_coverage:.1%}") # Item-level confidence scores for item in stmt.iter_with_values(): if hasattr(item, 'confidence') and item.confidence < 0.8: print(f"Low confidence: {item.label} ({item.confidence:.2f})")` Discovering Available Data -------------------------- Not sure what a company reports? Use the discovery methods to explore before querying: `facts = company.get_facts() # Search for concepts by keyword facts.search_concepts("revenue") # Find all revenue-related concepts facts.search_concepts("debt") # Find debt-related concepts # See what periods have data for a concept facts.available_periods("Revenue") # List all periods with Revenue data` These methods are especially useful when `get_fact()` returns `None` — the warnings will suggest using `search_concepts()` to find the right concept name and `available_periods()` to find valid periods. Both period formats work interchangeably: `"2023-FY"` and `"FY 2023"` are equivalent. Advanced Usage -------------- ### Working with EntityFacts Directly For advanced analysis, access the enhanced EntityFacts object with rich display: `facts = company.facts print(facts) # Rich console display with summary statistics and key metrics # Query specific facts with enhanced query interface revenue_facts = facts.query().by_concept('Revenue').execute() # Get time series for any concept revenue_ts = facts.time_series('Revenue', periods=20) # Get DEI (Document and Entity Information) facts dei_info = facts.dei_facts() entity_summary = facts.entity_info() # Generate comprehensive LLM context llm_context = facts.to_llm_context( focus_areas=['profitability', 'growth'], time_period='5Y' ) print(llm_context['focus_analysis']['profitability']) # Export as AI agent tools (MCP-compatible) agent_tools = facts.to_agent_tools() print(agent_tools[0]) # Tool definition for AI agents` Advanced Querying ----------------- The Facts API includes a powerful query interface for sophisticated financial analysis. Access it through the `query()` method: `facts = company.facts query = facts.query()` ### Basic Querying #### Filter by Concept `# Find all revenue-related facts revenue_facts = facts.query().by_concept('Revenue').execute() # Exact concept matching exact_revenue = facts.query().by_concept('us-gaap:Revenue', exact=True).execute() # Fuzzy matching (finds Revenue, Revenues, RevenueFromSales, etc.) revenue_like = facts.query().by_concept('revenue').execute()` #### Filter by Time Period `# Get facts from specific fiscal year fy2024_facts = facts.query().by_fiscal_year(2024).execute() # Get facts from specific quarter q1_facts = facts.query().by_fiscal_period('Q1').execute() # Get facts from date range from datetime import date recent_facts = facts.query().date_range( start=date(2023, 1, 1), end=date(2024, 12, 31) ).execute() # Get facts as of specific date (point-in-time) snapshot_facts = facts.query().as_of(date(2024, 6, 30)).execute()` #### Filter by Statement Type `# Income statement facts only income_facts = facts.query().by_statement_type('IncomeStatement').execute() # Balance sheet facts only balance_facts = facts.query().by_statement_type('BalanceSheet').execute() # Cash flow facts only cashflow_facts = facts.query().by_statement_type('CashFlow').execute()` #### Filter by Form Type `# Only audited annual facts (10-K forms) annual_facts = facts.query().by_form_type('10-K').execute() # Only quarterly facts (10-Q forms) quarterly_facts = facts.query().by_form_type('10-Q').execute() # Multiple form types periodic_facts = facts.query().by_form_type(['10-K', '10-Q']).execute()` ### Advanced Filtering #### Quality and Confidence Filters `# Only high-quality, audited facts high_quality = facts.query().high_quality_only().execute() # Facts above confidence threshold confident_facts = facts.query().min_confidence(0.9).execute()` #### Period Length Filtering `# Only quarterly periods (3 months) quarterly_only = facts.query().by_period_length(3).execute() # Only annual periods (12 months) annual_only = facts.query().by_period_length(12).execute() # Only year-to-date periods (9 months) ytd_facts = facts.query().by_period_length(9).execute()` #### Latest Facts `# Get most recent facts by filing date latest_facts = facts.query().by_concept('Revenue').latest(5) # Get latest instant facts (for balance sheet items) latest_balance = facts.query().by_statement_type('BalanceSheet').latest_instant().execute() # Get latest periods with preference latest_periods = facts.query().latest_periods(4, prefer_annual=True).execute()` ### Method Chaining Combine multiple filters for precise queries: `# Revenue facts from 2024 10-K filings only revenue_2024_annual = facts.query()\ .by_concept('Revenue')\ .by_fiscal_year(2024)\ .by_form_type('10-K')\ .execute() # High-quality quarterly income statement facts quality_quarterly = facts.query()\ .by_statement_type('IncomeStatement')\ .by_period_length(3)\ .high_quality_only()\ .execute() # Recent balance sheet facts as of year-end year_end_balance = facts.query()\ .by_statement_type('BalanceSheet')\ .as_of(date(2024, 12, 31))\ .latest_instant()\ .execute()` ### Output Formats #### Convert to DataFrame `# Basic DataFrame with all columns df = facts.query().by_concept('Revenue').to_dataframe() # DataFrame with selected columns df = facts.query().by_concept('Revenue').to_dataframe( 'label', 'numeric_value', 'fiscal_period', 'fiscal_year' ) print(df.head())` #### Pivot by Period Create time-series views with periods as columns: `# Get formatted financial statement stmt = facts.query()\ .by_statement_type('IncomeStatement')\ .latest_periods(4)\ .pivot_by_period() # Get raw DataFrame pivot pivot_df = facts.query()\ .by_statement_type('IncomeStatement')\ .latest_periods(4)\ .pivot_by_period(return_statement=False) print(pivot_df)` #### LLM-Ready Context `# Get facts in LLM-friendly format llm_context = facts.query().by_concept('Revenue').to_llm_context() # Perfect for feeding to AI models for fact_context in llm_context: print(f"Concept: {fact_context['concept']}") print(f"Value: {fact_context['value']}") print(f"Period: {fact_context['period']}")` ### Query Utilities #### Count Results `# Count matching facts without loading them revenue_count = facts.query().by_concept('Revenue').count() print(f"Found {revenue_count} revenue facts") # Enhanced query with rich display revenue_query = facts.query().by_concept('Revenue') print(revenue_query) # Rich representation of the query` #### Sort Results `# Sort by filing date (newest first) sorted_facts = facts.query()\ .by_concept('Revenue')\ .sort_by('filing_date', ascending=False)\ .execute() # Sort by fiscal year sorted_by_year = facts.query()\ .by_concept('Assets')\ .sort_by('fiscal_year')\ .execute()` ### Real-World Query Examples #### Track Revenue Growth Over Time `# Get quarterly revenue for trend analysis quarterly_revenue = facts.query()\ .by_concept('Revenue')\ .by_period_length(3)\ .sort_by('period_end')\ .to_dataframe('fiscal_year', 'fiscal_period', 'numeric_value', 'period_end') # Calculate quarter-over-quarter growth quarterly_revenue['growth'] = quarterly_revenue['numeric_value'].pct_change() * 100 print(quarterly_revenue[['fiscal_period', 'fiscal_year', 'numeric_value', 'growth']])` #### Compare Audited vs Unaudited Numbers `# Get both 10-K (audited) and 10-Q (unaudited) revenue for same period revenue_2024_q4 = facts.query()\ .by_concept('Revenue')\ .by_fiscal_year(2024)\ .by_fiscal_period('Q4')\ .by_form_type(['10-K', '10-Q'])\ .to_dataframe('form_type', 'numeric_value', 'filing_date') print(revenue_2024_q4)` #### Find Restatements `# Look for the same period filed multiple times eps_facts = facts.query()\ .by_concept('EarningsPerShare')\ .by_fiscal_year(2024)\ .by_fiscal_period('Q1')\ .sort_by('filing_date')\ .to_dataframe('filing_date', 'numeric_value', 'form_type') if len(eps_facts) > 1: print("Potential restatement found:") print(eps_facts)` #### Build Custom Financial Ratios `# Get components for current ratio calculation current_assets = facts.query()\ .by_concept('CurrentAssets')\ .latest_instant()\ .execute() current_liabilities = facts.query()\ .by_concept('CurrentLiabilities')\ .latest_instant()\ .execute() if current_assets and current_liabilities: assets_value = current_assets[0].numeric_value liabilities_value = current_liabilities[0].numeric_value current_ratio = assets_value / liabilities_value print(f"Current Ratio: {current_ratio:.2f}")` ### Query Performance Tips 1. **Use Specific Filters**: More specific queries run faster `# Good: Specific concept and year facts.query().by_concept('us-gaap:Revenue', exact=True).by_fiscal_year(2024) # Less efficient: Broad concept search facts.query().by_concept('revenue')` 2. **Limit Results Early**: Use `latest()` or `count()` when appropriate `# Good: Get just what you need recent_revenue = facts.query().by_concept('Revenue').latest(4) # Less efficient: Get all then slice all_revenue = facts.query().by_concept('Revenue').execute()[:4]` 3. **Chain Filters Logically**: Put most selective filters first `# Good: Narrow down quickly facts.query().by_fiscal_year(2024).by_form_type('10-K').by_concept('Revenue') # Less efficient: Broad filter first facts.query().by_concept('Revenue').by_fiscal_year(2024).by_form_type('10-K')` The query interface provides powerful flexibility for financial analysis while maintaining simplicity for common use cases. ### Enhanced Period Selection Logic The API intelligently handles period selection with improved consistency: `# Annual periods preferred - gets FY 2024, FY 2023, etc. annual = company.income_statement(annual=True) print(annual) # Rich display with period headers # Quarterly periods - gets most recent quarters quarterly = company.income_statement(annual=False) # Mixed periods automatically detected and handled mixed = company.income_statement(periods=8, annual=False) # API intelligently selects best available periods` **Enhanced Period Features:** * **Smart Labeling**: Periods labeled by fiscal quarters and years * **Consistency**: "Q2 2024" means period ending in company's fiscal Q2 of 2024 * **Hierarchy**: "FY 2024" means full fiscal year ending in 2024 * **Quality Indicators**: Period data quality shown in metadata * **Automatic Selection**: API selects best available periods when requested periods aren't available Error Handling -------------- The API is designed for graceful error handling: `company = Company('INVALIDTICKER') # These will return None instead of raising exceptions income_stmt = company.income_statement() # Returns None shares = company.shares_outstanding # Returns None facts = company.facts # Returns None # Check before using if company.facts: # Facts are available stmt = company.income_statement() else: print("No facts available for this company")` Real-World Examples ------------------- ### Compare Revenue Growth with Enhanced Display `from edgar import Company companies = ['AAPL', 'MSFT', 'GOOGL'] for ticker in companies: company = Company(ticker) if company.facts: # Get enhanced multi-period statement stmt = company.income_statement(periods=2) print(f"\n{ticker} Revenue Analysis:") print(stmt) # Rich multi-period display # Calculate growth using new methods df = stmt.to_dataframe() if not df.empty: revenue_row = df[df['label'].str.contains('Revenue', case=False, na=False)].iloc[0] periods = stmt.periods if len(periods) >= 2: current = revenue_row[periods[0]] prior = revenue_row[periods[1]] if current and prior: growth = ((current - prior) / prior) * 100 print(f"{ticker}: {growth:.1f}% revenue growth") # Generate LLM context for deeper analysis llm_data = stmt.to_llm_context() if 'key_metrics' in llm_data: print(f"AI Analysis Available: {list(llm_data['key_metrics'].keys())}") # Display some automatic calculations if 'profit_margin_fy_2024' in llm_data['key_metrics']: margin = llm_data['key_metrics']['profit_margin_fy_2024'] print(f"{ticker} Profit Margin: {margin:.1%}")` ### Build Enhanced Comparison Dashboard `import pandas as pd def compare_companies_enhanced(tickers, periods=2): results = [] for ticker in tickers: company = Company(ticker) if company.facts: # Get enhanced multi-period statement stmt = company.income_statement(periods=periods) # Extract LLM context for automated metrics llm_data = stmt.to_llm_context(include_metadata=True) # Build comprehensive comparison data company_data = { 'Company': company.name, 'Ticker': ticker, 'Periods': len(stmt.periods), 'Data_Quality': llm_data.get('metadata', {}).get('quality_indicators', []), } # Add revenue data for all periods revenue_item = stmt.find_item('Revenue') if revenue_item: for period in stmt.periods: value = revenue_item.values.get(period) if value: company_data[f'Revenue_{period.replace(" ", "_")}'] = value # Add key metrics if available if 'key_metrics' in llm_data: for metric, value in llm_data['key_metrics'].items(): company_data[f'Metric_{metric}'] = value results.append(company_data) return pd.DataFrame(results) # Compare with enhanced analytics comparison = compare_companies_enhanced(['AAPL', 'MSFT', 'GOOGL', 'AMZN']) print(comparison) # Web rendering example def render_for_web(ticker): company = Company(ticker) stmt = company.income_statement() web_data = [] for item in stmt.iter_with_values(): web_data.append({ 'concept': item.concept, 'label': item.label, 'depth': getattr(item, 'depth', 0), 'is_total': item.is_total, 'values': {period: item.get_display_value(period) for period in stmt.periods if item.values.get(period)} }) return web_data web_ready_data = render_for_web('AAPL') print(f"Generated {len(web_ready_data)} items for web display")` ### Extract Enhanced Key Metrics `def company_snapshot_enhanced(ticker): company = Company(ticker) snapshot = { 'name': company.name, 'ticker': ticker, 'shares_outstanding': company.shares_outstanding, 'public_float': company.public_float, 'has_facts': company.facts is not None } if company.facts: # Get entity information entity_info = company.facts.entity_info() snapshot.update(entity_info) # Get financial statement summaries with LLM context income_stmt = company.income_statement(periods=2) if income_stmt: llm_context = income_stmt.to_llm_context() snapshot.update({ 'revenue_latest': llm_context['data'].get('revenue_fy_2024') or llm_context['data'].get('revenue_q4_2024'), 'key_metrics': llm_context.get('key_metrics', {}), 'data_quality': llm_context.get('metadata', {}).get('quality_indicators', []) }) # Get balance sheet strength indicators balance_sheet = company.balance_sheet(periods=1) if balance_sheet: bs_context = balance_sheet.to_llm_context() assets_key = next((k for k in bs_context['data'].keys() if 'assets' in k.lower() and 'total' in k.lower()), None) if assets_key: snapshot['total_assets'] = bs_context['data'][assets_key] # Add balance sheet metrics if available if 'key_metrics' in bs_context: snapshot['balance_sheet_metrics'] = bs_context['key_metrics'] return snapshot # Get enhanced snapshots with auto-calculated metrics tickers = ['AAPL', 'TSLA', 'NVDA'] snapshots = [company_snapshot_enhanced(t) for t in tickers] df = pd.DataFrame(snapshots) print(df[['name', 'ticker', 'revenue_latest', 'total_assets']].to_string()) # Display detailed metrics for one company print("\nDetailed metrics for AAPL:") aapl_snapshot = snapshots[0] for key, value in aapl_snapshot.get('key_metrics', {}).items(): print(f"{key}: {value}") # Show data quality indicators if 'data_quality' in aapl_snapshot: print(f"Data Quality: {', '.join(aapl_snapshot['data_quality'])}") # Show balance sheet metrics if available if 'balance_sheet_metrics' in aapl_snapshot: print("\nBalance Sheet Metrics:") for key, value in aapl_snapshot['balance_sheet_metrics'].items(): print(f"{key}: {value}")` Performance Tips ---------------- 1. **Cache Company Objects**: Reuse Company instances to leverage enhanced caching 2. **Use as\_dataframe=True**: For bulk calculations, raw DataFrames are faster 3. **Limit Periods**: Request only the periods you need for analysis 4. **Check Availability**: Use `if company.facts:` before accessing financial data 5. **Choose Format Wisely**: Use `concise_format=True` for display, `False` for calculations 6. **Cache LLM Context**: Store `to_llm_context()` results for repeated AI analysis 7. **Batch Web Rendering**: Use `iter_with_values()` to skip empty items `# Good: Reuse company object with enhanced features company = Company('AAPL') if company.facts: print(company.facts) # Rich display with summary statistics # Get multiple statements efficiently income = company.income_statement() balance = company.balance_sheet() cash = company.cashflow_statement() # Cache LLM context for AI applications llm_context = income.to_llm_context() # Reuse llm_context for multiple AI queries # Good: Use DataFrame for bulk analysis df = company.income_statement(periods=10, as_dataframe=True) analysis = df.select_dtypes(include=[np.number]).pct_change() # Good: Efficient web rendering web_items = [item for item in stmt.iter_with_values()] # Only items with data rendered_data = stmt.to_dict() # Single conversion for web APIs # Good: Format choice based on use case exec_dashboard = company.income_statement(concise_format=True) # For presentations analysis_data = company.income_statement(concise_format=False) # For calculations` Integration with Other EdgarTools Features ------------------------------------------ The enhanced Facts API works seamlessly with other EdgarTools features: `company = Company('AAPL') # Combine with filings for comprehensive analysis latest_10k = company.latest('10-K') facts_stmt = company.income_statement() # Generate cross-referenced analysis analysis_context = { 'filing_info': { 'form': latest_10k.form, 'filing_date': latest_10k.filing_date, 'accession': latest_10k.accession_no }, 'financial_data': facts_stmt.to_llm_context(), 'data_sources': 'SEC Company Facts API + EDGAR Filings' } # Compare with traditional XBRL (if available) try: xbrl = latest_10k.xbrl() # Traditional XBRL approach xbrl_stmt = xbrl.statements.income_statement facts_stmt = company.income_statement() # Enhanced Facts API print("Data Source Comparison:") print(f"XBRL Concepts: {len(xbrl_stmt) if xbrl_stmt else 0}") print(f"Facts API Items: {len(facts_stmt.items)}") print(f"Facts API Quality: {getattr(facts_stmt, 'canonical_coverage', 'N/A')}") except: print("XBRL data not available - Facts API provides comprehensive coverage")` Migration Guide --------------- Upgrading from previous versions is straightforward with enhanced features: `# Previous approach (still works) old_facts = company.get_facts() # Returns basic format old_stmt = company.income_statement(as_dataframe=True) # Enhanced approach with new features facts = company.facts # Rich EntityFacts with console display stmt = company.income_statement() # MultiPeriodStatement with hierarchy # New formatting options compact_stmt = company.income_statement(concise_format=True) # $1.0B format full_stmt = company.income_statement(concise_format=False) # $1,000,000,000 format # New LLM integration llm_data = stmt.to_llm_context() # AI-ready structured data # New web integration web_items = list(stmt.iter_with_values()) # Easy web rendering specific_item = stmt.find_item('Revenue') # Direct item access # Enhanced property access with full context shares_fact = facts.shares_outstanding_fact # Full FinancialFact object shares_value = facts.shares_outstanding # Direct numeric value` **Key Improvements:** * **Backward Compatible**: All existing code continues to work * **Enhanced Display**: Rich console formatting with colors and hierarchy * **Better Formatting**: Smart value formatting with concise options * **AI Integration**: Built-in LLM context generation * **Web Support**: Easy iteration and rendering methods * **Performance**: Optimized caching and data structures Troubleshooting --------------- **Q: `get_fact()` or `get_concept()` returned None — how do I find the right concept?** A: These methods now emit a warning when a concept is not found, including suggestions for similar concept names. Use `search_concepts()` to find what the company actually reports, and `available_periods()` to see what periods have data: `facts = company.get_facts() facts.search_concepts("revenue") # Shows all revenue-related concepts facts.available_periods("Revenue") # Shows periods with Revenue data` **Q: Why do some companies return None for financial statements?** A: Not all companies have facts data available through the SEC API. This is normal for some entity types. The enhanced API provides better error handling and fallback strategies. **Q: What's the difference between concise\_format=True and False?** A: `concise_format=False` (default) shows full numbers with commas ($1,000,000,000) for precision. `concise_format=True` shows scaled format ($1.0B) for presentations. Per-share amounts are always decimals regardless of setting. **Q: How do I use the hierarchical structure in web applications?** A: Use the iteration methods: `stmt.iter_hierarchy()` for parent-child relationships, `stmt.iter_with_values()` for items with data, or `stmt.to_dict()` for nested JSON structure. **Q: How do I get the most recent quarter with the new format?** A: Use `company.income_statement(periods=1, annual=False)` to get the latest quarterly period with enhanced formatting and hierarchy. **Q: Can I get historical data beyond what's shown?** A: Yes, increase the `periods` parameter: `company.income_statement(periods=20)` for extensive historical data with consistent formatting. **Q: How do I integrate with AI/LLM applications?** A: Use `stmt.to_llm_context()` to get structured, AI-ready data with automatic metric calculations and clean formatting optimized for language models. API Reference ------------- ### MultiPeriodStatement Methods | Method | Description | | --- | --- | | `to_dataframe()` | Convert to pandas DataFrame with metadata | | `to_llm_context()` | Generate AI-ready structured context | | `iter_hierarchy()` | Iterate with depth and parent information | | `iter_with_values()` | Iterate only items with values | | `find_item(concept)` | Find specific item by concept or label | | `to_dict()` | Convert to nested dictionary structure | | `to_flat_list()` | Convert to flat list for web APIs | | `get_period_comparison()` | Get period-over-period analysis | ### EntityFacts Enhanced Methods | Method | Description | | --- | --- | | `to_llm_context()` | Comprehensive AI context with focus areas | | `to_agent_tools()` | Export as MCP-compatible agent tools | | `calculate_ratios()` | Financial ratio calculations | | `peer_comparison()` | Compare with peer companies | | `detect_anomalies()` | Identify unusual patterns | ### New Parameters | Parameter | Methods | Description | | --- | --- | --- | | `concise_format` | All statement methods | Value display format control | | `include_metadata` | `to_llm_context()` | Include data quality metrics | | `flatten_values` | `to_llm_context()` | Flatten multi-period values | | `focus_areas` | `to_llm_context()` | Emphasize specific analysis areas | For complete API documentation of the underlying EntityFacts class and query interface, see the [EntityFacts API Reference](https://edgartools.readthedocs.io/en/latest/api/entity-facts-reference/) . * * * _The enhanced Company Facts API is part of EdgarTools' comprehensive SEC data platform, now with AI integration, web rendering support, and professional formatting. For more information, visit the [EdgarTools Documentation](https://edgartools.dev/) ._ Back to top --- # Company Facts - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/company-facts/#company-facts-query-historical-sec-financial-data-with-python) Company Facts: Query Historical SEC Financial Data with Python ============================================================== The Company Facts API provides comprehensive access to SEC financial data through an intuitive, AI-ready interface. Get financial statements, key metrics, and detailed company information with just a few lines of code. ✨ **Latest Features:** * **Enhanced Value Formatting**: Full numbers with commas (1,000,000,000) by default, with optional concise format ($1.0B) * **Multi-Period Statements**: Rich hierarchical display showing multiple periods side-by-side * **LLM Integration**: Built-in `to_llm_context()` method for AI consumption * **Web Rendering Support**: Easy iteration over statement items with comprehensive web API methods * **Improved Visual Display**: Professional formatting with color-coded values and hierarchical structure Quick Start ----------- `from edgar import Company # Get any public company company = Company('AAPL') # Ticker symbol # or company = Company(320193) # CIK number # Access key metrics instantly print(f"Shares Outstanding: {company.shares_outstanding:,.0f}") print(f"Public Float: ${company.public_float:,.0f}") # Get enhanced multi-period financial statements income_stmt = company.income_statement() # Shows multiple periods with hierarchy balance_sheet = company.balance_sheet() cash_flow = company.cashflow_statement() print(income_stmt) # Rich multi-period display # Get concise format for quick overview income_compact = company.income_statement(concise_format=True) print(income_compact) # Shows $1.0B instead of $1,000,000,000` Key Features ------------ * **🚀 Zero Setup** - Works immediately with existing Company objects * **💰 Full Precision** - Full numbers with commas by default, optional concise formatting * **📊 Enhanced Display** - Multi-period hierarchical statements with rich formatting * **🛡️ Error Resilient** - Graceful handling of missing data with intelligent fallbacks * **🤖 AI-Ready** - Built-in LLM context generation with structured data output * **🌐 Web Integration** - Easy iteration methods and rendering support for web applications * **⚡ Performance Optimized** - Intelligent caching and efficient data structures * **🎨 Professional Formatting** - Color-coded values, hierarchical structure, and smart spacing Core Properties --------------- ### Company Metrics Access essential company information through simple properties: `company = Company('TSLA') # Key financial metrics print(f"Shares Outstanding: {company.shares_outstanding:,.0f}") print(f"Public Float: ${company.public_float:,.0f}") # Check if facts are available if company.facts: print(f"Total facts available: {len(company.facts):,}")` **Available Properties:** * `company.facts` - Access to the full EntityFacts object * `company.shares_outstanding` - Number of shares outstanding * `company.public_float` - Public float value in dollars Financial Statements -------------------- ### Income Statement Get hierarchical income statement data with flexible period options: `# Default: 4 annual periods, enhanced multi-period display income_stmt = company.income_statement() print(income_stmt) # Rich hierarchical display with multiple periods # Get 8 quarterly periods with full number formatting quarterly = company.income_statement(periods=8, annual=False) # Use concise format for quick analysis ($1.0B vs $1,000,000,000) compact = company.income_statement(concise_format=True) # Get raw DataFrame for analysis df = company.income_statement(periods=4, as_dataframe=True) # Convert to LLM-friendly format llm_data = income_stmt.to_llm_context() print(llm_data['key_metrics']) # Automatic ratio calculations` ### Balance Sheet Access hierarchical balance sheet data for point-in-time or trend analysis: `# Enhanced multi-period balance sheet with hierarchy balance_sheet = company.balance_sheet(periods=4) print(balance_sheet) # Shows Assets, Liabilities, Equity sections # Point-in-time snapshot as of specific date from datetime import date snapshot = company.balance_sheet(as_of=date(2024, 12, 31)) # Concise format for executive summaries exec_summary = company.balance_sheet(concise_format=True) # Raw data for calculations df = company.balance_sheet(periods=3, as_dataframe=True) # Web rendering support - iterate over items for item in balance_sheet: print(f"{item.label}: {item.get_display_value(balance_sheet.periods[0])}")` ### Cash Flow Statement Analyze hierarchical cash flow patterns across periods: `# Enhanced annual cash flow with operating/investing/financing sections cash_flow = company.cashflow_statement(periods=5, annual=True) print(cash_flow) # Rich display with cash flow categories # Quarterly cash flow analysis with full formatting quarterly_cf = company.cashflow_statement(periods=8, annual=False) # Executive dashboard format exec_cf = company.cashflow_statement(concise_format=True) # Generate analysis context for AI ai_context = cash_flow.to_llm_context(include_metadata=True) print(ai_context['key_metrics']) # Automatic cash flow metrics` Method Parameters ----------------- All financial statement methods support consistent parameters: | Parameter | Type | Default | Description | | --- | --- | --- | --- | | `periods` | int | 4 | Number of periods to retrieve | | `annual` | bool | True | If True, prefer annual periods; if False, get quarterly | | `as_dataframe` | bool | False | If True, return raw DataFrame; if False, return MultiPeriodStatement | | `concise_format` | bool | False | If True, display as $1.0B; if False, display as $1,000,000,000 | **Special Parameters:** - `balance_sheet()` also supports `as_of` parameter for point-in-time views Return Types ------------ ### MultiPeriodStatement Objects (Default) When `as_dataframe=False` (default), methods return enhanced `MultiPeriodStatement` objects with: * **Hierarchical Structure**: Organized sections with proper parent-child relationships * **Multi-Period Display**: Side-by-side period comparison with rich formatting * **Smart Value Formatting**: Full numbers ($1,000,000,000) by default, per-share amounts as decimals * **Color-Coded Display**: Green/red values, bold totals, hierarchical indentation * **Web Rendering Support**: Easy iteration and item access for web applications * **LLM Integration**: Built-in context generation for AI analysis `stmt = company.income_statement() # Rich multi-period display (automatic in notebooks) print(stmt) # Convert to DataFrame for analysis df = stmt.to_dataframe() revenue_growth = df.loc['Revenue'].pct_change() # Generate LLM-friendly context llm_data = stmt.to_llm_context() print(llm_data['key_metrics']['profit_margin_fy_2024']) # Iterate over items for web rendering for item in stmt.iter_with_values(): print(f"{item.label}: {item.get_display_value(stmt.periods[0])}") # Get specific item revenue_item = stmt.find_item('Revenue') if revenue_item: print(f"Revenue trend: {revenue_item.values}")` ### DataFrame Objects When `as_dataframe=True`, methods return pandas DataFrames with enhanced structure: `df = company.income_statement(as_dataframe=True) # Enhanced DataFrame with metadata columns print(df.columns) # Includes: periods, depth, is_total, section, confidence print(df.dtypes) print(df.describe()) # Access financial data revenue_series = df.loc['us-gaap:Revenues'] # Full concept names as index print(df[df['is_total']]) # Filter to total/subtotal rows only print(df[df['section'] == 'Revenue']) # Filter by statement section` Enhanced Features ----------------- ### Value Formatting Options The API now provides flexible value formatting to suit different use cases: `# Full precision formatting (default) - best for analysis stmt_full = company.income_statement(concise_format=False) print(stmt_full) # Shows: $391,035,000,000 # Concise formatting - best for presentations and dashboards stmt_concise = company.income_statement(concise_format=True) print(stmt_concise) # Shows: $391.0B # Per-share amounts are always displayed as decimals # Example: Earnings Per Share shows as "2.97" not "$2.97" or "$2,970,000,000"` **Formatting Rules:** * **Default (`concise_format=False`)**: Full numbers with commas ($1,000,000,000) * **Concise (`concise_format=True`)**: Scaled format ($1.0B, $500.3M) * **Per-Share Values**: Always decimal format (2.97) regardless of setting * **Negative Values**: Properly formatted with minus signs * **Zero/Null Values**: Displayed as "-" for clean presentation ### LLM Integration and AI Context Generate structured data optimized for AI and LLM consumption: `stmt = company.income_statement(periods=4) # Generate LLM-friendly context llm_context = stmt.to_llm_context( include_metadata=True, # Include data quality metrics include_hierarchy=False, # Flatten for simplicity (default) flatten_values=True # Create period-prefixed keys (default) ) print("LLM Context Structure:") print(f"Company: {llm_context['company']}") print(f"Statement Type: {llm_context['statement_type']}") print(f"Periods: {llm_context['periods']}") print(f"Data Quality: {llm_context['metadata']['quality_indicators']}") # Access flattened financial data financial_data = llm_context['data'] print(f"Revenue FY 2024: ${financial_data.get('revenue_fy_2024', 0):,.0f}") print(f"Revenue FY 2023: ${financial_data.get('revenue_fy_2023', 0):,.0f}") # Automatic ratio calculations key_metrics = llm_context.get('key_metrics', {}) if 'profit_margin_fy_2024' in key_metrics: print(f"Current Profit Margin: {key_metrics['profit_margin_fy_2024']:.1%}") # Feed to LLM for analysis import json analysis_prompt = f""" Analyze this financial data for {llm_context['company']}: {json.dumps(llm_context, indent=2)} Provide insights on profitability trends and growth patterns. """` ### Web Application Integration Easy iteration and rendering support for web applications: `stmt = company.income_statement(periods=4) # Basic iteration over all items for item in stmt: print(f"{item.label}: {item.get_display_value(stmt.periods[0])}") # Iterate with hierarchy information for item in stmt.iter_hierarchy(): indent = " " * item.depth parent_info = f" (parent: {item.parent.label})" if item.parent else "" print(f"{indent}{item.label}{parent_info}") # Only items with values (skip empty rows) for item in stmt.iter_with_values(): values_summary = ", ".join([ f"{period}: {item.get_display_value(period)}" for period in stmt.periods if item.values.get(period) ]) print(f"{item.label} -> {values_summary}") # Find specific items revenue_item = stmt.find_item('Revenue') if revenue_item: print(f"Found Revenue: {revenue_item.values}") # Convert to web-friendly format web_data = stmt.to_dict() # Nested dictionary flat_data = stmt.to_flat_list() # Flat list for tables # Period comparison analysis comparison = stmt.get_period_comparison() for concept, analysis in comparison.items(): if analysis['growth_rate']: print(f"{concept}: {analysis['growth_rate']:.1%} growth")` ### Advanced Statement Features #### Smart Hierarchical Organization Statements now display with intelligent hierarchy based on accounting standards: `stmt = company.income_statement() print(stmt) # Shows: # Revenue # Product Revenue # Service Revenue # Cost of Revenue # Cost of Product Sales # Cost of Services # Gross Profit [calculated] # Operating Expenses # Research and Development # Sales and Marketing # Operating Income [calculated]` #### Professional Visual Display * **Color Coding**: Green for positive values, red for negative * **Bold Formatting**: Totals and subtotals are emphasized * **Hierarchical Indentation**: Clear parent-child relationships * **Confidence Indicators**: Low-confidence items marked with ◦ * **Smart Spacing**: Separators after major sections #### Enhanced Data Quality Statements include data quality metadata: `stmt = company.income_statement() # Check overall statement quality if hasattr(stmt, 'canonical_coverage'): print(f"Canonical Coverage: {stmt.canonical_coverage:.1%}") # Item-level confidence scores for item in stmt.iter_with_values(): if hasattr(item, 'confidence') and item.confidence < 0.8: print(f"Low confidence: {item.label} ({item.confidence:.2f})")` Discovering Available Data -------------------------- Not sure what a company reports? Use the discovery methods to explore before querying: `facts = company.get_facts() # Search for concepts by keyword facts.search_concepts("revenue") # Find all revenue-related concepts facts.search_concepts("debt") # Find debt-related concepts # See what periods have data for a concept facts.available_periods("Revenue") # List all periods with Revenue data` These methods are especially useful when `get_fact()` returns `None` — the warnings will suggest using `search_concepts()` to find the right concept name and `available_periods()` to find valid periods. Both period formats work interchangeably: `"2023-FY"` and `"FY 2023"` are equivalent. Advanced Usage -------------- ### Working with EntityFacts Directly For advanced analysis, access the enhanced EntityFacts object with rich display: `facts = company.facts print(facts) # Rich console display with summary statistics and key metrics # Query specific facts with enhanced query interface revenue_facts = facts.query().by_concept('Revenue').execute() # Get time series for any concept revenue_ts = facts.time_series('Revenue', periods=20) # Get DEI (Document and Entity Information) facts dei_info = facts.dei_facts() entity_summary = facts.entity_info() # Generate comprehensive LLM context llm_context = facts.to_llm_context( focus_areas=['profitability', 'growth'], time_period='5Y' ) print(llm_context['focus_analysis']['profitability']) # Export as AI agent tools (MCP-compatible) agent_tools = facts.to_agent_tools() print(agent_tools[0]) # Tool definition for AI agents` Advanced Querying ----------------- The Facts API includes a powerful query interface for sophisticated financial analysis. Access it through the `query()` method: `facts = company.facts query = facts.query()` ### Basic Querying #### Filter by Concept `# Find all revenue-related facts revenue_facts = facts.query().by_concept('Revenue').execute() # Exact concept matching exact_revenue = facts.query().by_concept('us-gaap:Revenue', exact=True).execute() # Fuzzy matching (finds Revenue, Revenues, RevenueFromSales, etc.) revenue_like = facts.query().by_concept('revenue').execute()` #### Filter by Time Period `# Get facts from specific fiscal year fy2024_facts = facts.query().by_fiscal_year(2024).execute() # Get facts from specific quarter q1_facts = facts.query().by_fiscal_period('Q1').execute() # Get facts from date range from datetime import date recent_facts = facts.query().date_range( start=date(2023, 1, 1), end=date(2024, 12, 31) ).execute() # Get facts as of specific date (point-in-time) snapshot_facts = facts.query().as_of(date(2024, 6, 30)).execute()` #### Filter by Statement Type `# Income statement facts only income_facts = facts.query().by_statement_type('IncomeStatement').execute() # Balance sheet facts only balance_facts = facts.query().by_statement_type('BalanceSheet').execute() # Cash flow facts only cashflow_facts = facts.query().by_statement_type('CashFlow').execute()` #### Filter by Form Type `# Only audited annual facts (10-K forms) annual_facts = facts.query().by_form_type('10-K').execute() # Only quarterly facts (10-Q forms) quarterly_facts = facts.query().by_form_type('10-Q').execute() # Multiple form types periodic_facts = facts.query().by_form_type(['10-K', '10-Q']).execute()` ### Advanced Filtering #### Quality and Confidence Filters `# Only high-quality, audited facts high_quality = facts.query().high_quality_only().execute() # Facts above confidence threshold confident_facts = facts.query().min_confidence(0.9).execute()` #### Period Length Filtering `# Only quarterly periods (3 months) quarterly_only = facts.query().by_period_length(3).execute() # Only annual periods (12 months) annual_only = facts.query().by_period_length(12).execute() # Only year-to-date periods (9 months) ytd_facts = facts.query().by_period_length(9).execute()` #### Latest Facts `# Get most recent facts by filing date latest_facts = facts.query().by_concept('Revenue').latest(5) # Get latest instant facts (for balance sheet items) latest_balance = facts.query().by_statement_type('BalanceSheet').latest_instant().execute() # Get latest periods with preference latest_periods = facts.query().latest_periods(4, prefer_annual=True).execute()` ### Method Chaining Combine multiple filters for precise queries: `# Revenue facts from 2024 10-K filings only revenue_2024_annual = facts.query()\ .by_concept('Revenue')\ .by_fiscal_year(2024)\ .by_form_type('10-K')\ .execute() # High-quality quarterly income statement facts quality_quarterly = facts.query()\ .by_statement_type('IncomeStatement')\ .by_period_length(3)\ .high_quality_only()\ .execute() # Recent balance sheet facts as of year-end year_end_balance = facts.query()\ .by_statement_type('BalanceSheet')\ .as_of(date(2024, 12, 31))\ .latest_instant()\ .execute()` ### Output Formats #### Convert to DataFrame `# Basic DataFrame with all columns df = facts.query().by_concept('Revenue').to_dataframe() # DataFrame with selected columns df = facts.query().by_concept('Revenue').to_dataframe( 'label', 'numeric_value', 'fiscal_period', 'fiscal_year' ) print(df.head())` #### Pivot by Period Create time-series views with periods as columns: `# Get formatted financial statement stmt = facts.query()\ .by_statement_type('IncomeStatement')\ .latest_periods(4)\ .pivot_by_period() # Get raw DataFrame pivot pivot_df = facts.query()\ .by_statement_type('IncomeStatement')\ .latest_periods(4)\ .pivot_by_period(return_statement=False) print(pivot_df)` #### LLM-Ready Context `# Get facts in LLM-friendly format llm_context = facts.query().by_concept('Revenue').to_llm_context() # Perfect for feeding to AI models for fact_context in llm_context: print(f"Concept: {fact_context['concept']}") print(f"Value: {fact_context['value']}") print(f"Period: {fact_context['period']}")` ### Query Utilities #### Count Results `# Count matching facts without loading them revenue_count = facts.query().by_concept('Revenue').count() print(f"Found {revenue_count} revenue facts") # Enhanced query with rich display revenue_query = facts.query().by_concept('Revenue') print(revenue_query) # Rich representation of the query` #### Sort Results `# Sort by filing date (newest first) sorted_facts = facts.query()\ .by_concept('Revenue')\ .sort_by('filing_date', ascending=False)\ .execute() # Sort by fiscal year sorted_by_year = facts.query()\ .by_concept('Assets')\ .sort_by('fiscal_year')\ .execute()` ### Real-World Query Examples #### Track Revenue Growth Over Time `# Get quarterly revenue for trend analysis quarterly_revenue = facts.query()\ .by_concept('Revenue')\ .by_period_length(3)\ .sort_by('period_end')\ .to_dataframe('fiscal_year', 'fiscal_period', 'numeric_value', 'period_end') # Calculate quarter-over-quarter growth quarterly_revenue['growth'] = quarterly_revenue['numeric_value'].pct_change() * 100 print(quarterly_revenue[['fiscal_period', 'fiscal_year', 'numeric_value', 'growth']])` #### Compare Audited vs Unaudited Numbers `# Get both 10-K (audited) and 10-Q (unaudited) revenue for same period revenue_2024_q4 = facts.query()\ .by_concept('Revenue')\ .by_fiscal_year(2024)\ .by_fiscal_period('Q4')\ .by_form_type(['10-K', '10-Q'])\ .to_dataframe('form_type', 'numeric_value', 'filing_date') print(revenue_2024_q4)` #### Find Restatements `# Look for the same period filed multiple times eps_facts = facts.query()\ .by_concept('EarningsPerShare')\ .by_fiscal_year(2024)\ .by_fiscal_period('Q1')\ .sort_by('filing_date')\ .to_dataframe('filing_date', 'numeric_value', 'form_type') if len(eps_facts) > 1: print("Potential restatement found:") print(eps_facts)` #### Build Custom Financial Ratios `# Get components for current ratio calculation current_assets = facts.query()\ .by_concept('CurrentAssets')\ .latest_instant()\ .execute() current_liabilities = facts.query()\ .by_concept('CurrentLiabilities')\ .latest_instant()\ .execute() if current_assets and current_liabilities: assets_value = current_assets[0].numeric_value liabilities_value = current_liabilities[0].numeric_value current_ratio = assets_value / liabilities_value print(f"Current Ratio: {current_ratio:.2f}")` ### Query Performance Tips 1. **Use Specific Filters**: More specific queries run faster `# Good: Specific concept and year facts.query().by_concept('us-gaap:Revenue', exact=True).by_fiscal_year(2024) # Less efficient: Broad concept search facts.query().by_concept('revenue')` 2. **Limit Results Early**: Use `latest()` or `count()` when appropriate `# Good: Get just what you need recent_revenue = facts.query().by_concept('Revenue').latest(4) # Less efficient: Get all then slice all_revenue = facts.query().by_concept('Revenue').execute()[:4]` 3. **Chain Filters Logically**: Put most selective filters first `# Good: Narrow down quickly facts.query().by_fiscal_year(2024).by_form_type('10-K').by_concept('Revenue') # Less efficient: Broad filter first facts.query().by_concept('Revenue').by_fiscal_year(2024).by_form_type('10-K')` The query interface provides powerful flexibility for financial analysis while maintaining simplicity for common use cases. ### Enhanced Period Selection Logic The API intelligently handles period selection with improved consistency: `# Annual periods preferred - gets FY 2024, FY 2023, etc. annual = company.income_statement(annual=True) print(annual) # Rich display with period headers # Quarterly periods - gets most recent quarters quarterly = company.income_statement(annual=False) # Mixed periods automatically detected and handled mixed = company.income_statement(periods=8, annual=False) # API intelligently selects best available periods` **Enhanced Period Features:** * **Smart Labeling**: Periods labeled by fiscal quarters and years * **Consistency**: "Q2 2024" means period ending in company's fiscal Q2 of 2024 * **Hierarchy**: "FY 2024" means full fiscal year ending in 2024 * **Quality Indicators**: Period data quality shown in metadata * **Automatic Selection**: API selects best available periods when requested periods aren't available Error Handling -------------- The API is designed for graceful error handling: `company = Company('INVALIDTICKER') # These will return None instead of raising exceptions income_stmt = company.income_statement() # Returns None shares = company.shares_outstanding # Returns None facts = company.facts # Returns None # Check before using if company.facts: # Facts are available stmt = company.income_statement() else: print("No facts available for this company")` Real-World Examples ------------------- ### Compare Revenue Growth with Enhanced Display `from edgar import Company companies = ['AAPL', 'MSFT', 'GOOGL'] for ticker in companies: company = Company(ticker) if company.facts: # Get enhanced multi-period statement stmt = company.income_statement(periods=2) print(f"\n{ticker} Revenue Analysis:") print(stmt) # Rich multi-period display # Calculate growth using new methods df = stmt.to_dataframe() if not df.empty: revenue_row = df[df['label'].str.contains('Revenue', case=False, na=False)].iloc[0] periods = stmt.periods if len(periods) >= 2: current = revenue_row[periods[0]] prior = revenue_row[periods[1]] if current and prior: growth = ((current - prior) / prior) * 100 print(f"{ticker}: {growth:.1f}% revenue growth") # Generate LLM context for deeper analysis llm_data = stmt.to_llm_context() if 'key_metrics' in llm_data: print(f"AI Analysis Available: {list(llm_data['key_metrics'].keys())}") # Display some automatic calculations if 'profit_margin_fy_2024' in llm_data['key_metrics']: margin = llm_data['key_metrics']['profit_margin_fy_2024'] print(f"{ticker} Profit Margin: {margin:.1%}")` ### Build Enhanced Comparison Dashboard `import pandas as pd def compare_companies_enhanced(tickers, periods=2): results = [] for ticker in tickers: company = Company(ticker) if company.facts: # Get enhanced multi-period statement stmt = company.income_statement(periods=periods) # Extract LLM context for automated metrics llm_data = stmt.to_llm_context(include_metadata=True) # Build comprehensive comparison data company_data = { 'Company': company.name, 'Ticker': ticker, 'Periods': len(stmt.periods), 'Data_Quality': llm_data.get('metadata', {}).get('quality_indicators', []), } # Add revenue data for all periods revenue_item = stmt.find_item('Revenue') if revenue_item: for period in stmt.periods: value = revenue_item.values.get(period) if value: company_data[f'Revenue_{period.replace(" ", "_")}'] = value # Add key metrics if available if 'key_metrics' in llm_data: for metric, value in llm_data['key_metrics'].items(): company_data[f'Metric_{metric}'] = value results.append(company_data) return pd.DataFrame(results) # Compare with enhanced analytics comparison = compare_companies_enhanced(['AAPL', 'MSFT', 'GOOGL', 'AMZN']) print(comparison) # Web rendering example def render_for_web(ticker): company = Company(ticker) stmt = company.income_statement() web_data = [] for item in stmt.iter_with_values(): web_data.append({ 'concept': item.concept, 'label': item.label, 'depth': getattr(item, 'depth', 0), 'is_total': item.is_total, 'values': {period: item.get_display_value(period) for period in stmt.periods if item.values.get(period)} }) return web_data web_ready_data = render_for_web('AAPL') print(f"Generated {len(web_ready_data)} items for web display")` ### Extract Enhanced Key Metrics `def company_snapshot_enhanced(ticker): company = Company(ticker) snapshot = { 'name': company.name, 'ticker': ticker, 'shares_outstanding': company.shares_outstanding, 'public_float': company.public_float, 'has_facts': company.facts is not None } if company.facts: # Get entity information entity_info = company.facts.entity_info() snapshot.update(entity_info) # Get financial statement summaries with LLM context income_stmt = company.income_statement(periods=2) if income_stmt: llm_context = income_stmt.to_llm_context() snapshot.update({ 'revenue_latest': llm_context['data'].get('revenue_fy_2024') or llm_context['data'].get('revenue_q4_2024'), 'key_metrics': llm_context.get('key_metrics', {}), 'data_quality': llm_context.get('metadata', {}).get('quality_indicators', []) }) # Get balance sheet strength indicators balance_sheet = company.balance_sheet(periods=1) if balance_sheet: bs_context = balance_sheet.to_llm_context() assets_key = next((k for k in bs_context['data'].keys() if 'assets' in k.lower() and 'total' in k.lower()), None) if assets_key: snapshot['total_assets'] = bs_context['data'][assets_key] # Add balance sheet metrics if available if 'key_metrics' in bs_context: snapshot['balance_sheet_metrics'] = bs_context['key_metrics'] return snapshot # Get enhanced snapshots with auto-calculated metrics tickers = ['AAPL', 'TSLA', 'NVDA'] snapshots = [company_snapshot_enhanced(t) for t in tickers] df = pd.DataFrame(snapshots) print(df[['name', 'ticker', 'revenue_latest', 'total_assets']].to_string()) # Display detailed metrics for one company print("\nDetailed metrics for AAPL:") aapl_snapshot = snapshots[0] for key, value in aapl_snapshot.get('key_metrics', {}).items(): print(f"{key}: {value}") # Show data quality indicators if 'data_quality' in aapl_snapshot: print(f"Data Quality: {', '.join(aapl_snapshot['data_quality'])}") # Show balance sheet metrics if available if 'balance_sheet_metrics' in aapl_snapshot: print("\nBalance Sheet Metrics:") for key, value in aapl_snapshot['balance_sheet_metrics'].items(): print(f"{key}: {value}")` Performance Tips ---------------- 1. **Cache Company Objects**: Reuse Company instances to leverage enhanced caching 2. **Use as\_dataframe=True**: For bulk calculations, raw DataFrames are faster 3. **Limit Periods**: Request only the periods you need for analysis 4. **Check Availability**: Use `if company.facts:` before accessing financial data 5. **Choose Format Wisely**: Use `concise_format=True` for display, `False` for calculations 6. **Cache LLM Context**: Store `to_llm_context()` results for repeated AI analysis 7. **Batch Web Rendering**: Use `iter_with_values()` to skip empty items `# Good: Reuse company object with enhanced features company = Company('AAPL') if company.facts: print(company.facts) # Rich display with summary statistics # Get multiple statements efficiently income = company.income_statement() balance = company.balance_sheet() cash = company.cashflow_statement() # Cache LLM context for AI applications llm_context = income.to_llm_context() # Reuse llm_context for multiple AI queries # Good: Use DataFrame for bulk analysis df = company.income_statement(periods=10, as_dataframe=True) analysis = df.select_dtypes(include=[np.number]).pct_change() # Good: Efficient web rendering web_items = [item for item in stmt.iter_with_values()] # Only items with data rendered_data = stmt.to_dict() # Single conversion for web APIs # Good: Format choice based on use case exec_dashboard = company.income_statement(concise_format=True) # For presentations analysis_data = company.income_statement(concise_format=False) # For calculations` Integration with Other EdgarTools Features ------------------------------------------ The enhanced Facts API works seamlessly with other EdgarTools features: `company = Company('AAPL') # Combine with filings for comprehensive analysis latest_10k = company.latest('10-K') facts_stmt = company.income_statement() # Generate cross-referenced analysis analysis_context = { 'filing_info': { 'form': latest_10k.form, 'filing_date': latest_10k.filing_date, 'accession': latest_10k.accession_no }, 'financial_data': facts_stmt.to_llm_context(), 'data_sources': 'SEC Company Facts API + EDGAR Filings' } # Compare with traditional XBRL (if available) try: xbrl = latest_10k.xbrl() # Traditional XBRL approach xbrl_stmt = xbrl.statements.income_statement facts_stmt = company.income_statement() # Enhanced Facts API print("Data Source Comparison:") print(f"XBRL Concepts: {len(xbrl_stmt) if xbrl_stmt else 0}") print(f"Facts API Items: {len(facts_stmt.items)}") print(f"Facts API Quality: {getattr(facts_stmt, 'canonical_coverage', 'N/A')}") except: print("XBRL data not available - Facts API provides comprehensive coverage")` Migration Guide --------------- Upgrading from previous versions is straightforward with enhanced features: `# Previous approach (still works) old_facts = company.get_facts() # Returns basic format old_stmt = company.income_statement(as_dataframe=True) # Enhanced approach with new features facts = company.facts # Rich EntityFacts with console display stmt = company.income_statement() # MultiPeriodStatement with hierarchy # New formatting options compact_stmt = company.income_statement(concise_format=True) # $1.0B format full_stmt = company.income_statement(concise_format=False) # $1,000,000,000 format # New LLM integration llm_data = stmt.to_llm_context() # AI-ready structured data # New web integration web_items = list(stmt.iter_with_values()) # Easy web rendering specific_item = stmt.find_item('Revenue') # Direct item access # Enhanced property access with full context shares_fact = facts.shares_outstanding_fact # Full FinancialFact object shares_value = facts.shares_outstanding # Direct numeric value` **Key Improvements:** * **Backward Compatible**: All existing code continues to work * **Enhanced Display**: Rich console formatting with colors and hierarchy * **Better Formatting**: Smart value formatting with concise options * **AI Integration**: Built-in LLM context generation * **Web Support**: Easy iteration and rendering methods * **Performance**: Optimized caching and data structures Troubleshooting --------------- **Q: `get_fact()` or `get_concept()` returned None — how do I find the right concept?** A: These methods now emit a warning when a concept is not found, including suggestions for similar concept names. Use `search_concepts()` to find what the company actually reports, and `available_periods()` to see what periods have data: `facts = company.get_facts() facts.search_concepts("revenue") # Shows all revenue-related concepts facts.available_periods("Revenue") # Shows periods with Revenue data` **Q: Why do some companies return None for financial statements?** A: Not all companies have facts data available through the SEC API. This is normal for some entity types. The enhanced API provides better error handling and fallback strategies. **Q: What's the difference between concise\_format=True and False?** A: `concise_format=False` (default) shows full numbers with commas ($1,000,000,000) for precision. `concise_format=True` shows scaled format ($1.0B) for presentations. Per-share amounts are always decimals regardless of setting. **Q: How do I use the hierarchical structure in web applications?** A: Use the iteration methods: `stmt.iter_hierarchy()` for parent-child relationships, `stmt.iter_with_values()` for items with data, or `stmt.to_dict()` for nested JSON structure. **Q: How do I get the most recent quarter with the new format?** A: Use `company.income_statement(periods=1, annual=False)` to get the latest quarterly period with enhanced formatting and hierarchy. **Q: Can I get historical data beyond what's shown?** A: Yes, increase the `periods` parameter: `company.income_statement(periods=20)` for extensive historical data with consistent formatting. **Q: How do I integrate with AI/LLM applications?** A: Use `stmt.to_llm_context()` to get structured, AI-ready data with automatic metric calculations and clean formatting optimized for language models. API Reference ------------- ### MultiPeriodStatement Methods | Method | Description | | --- | --- | | `to_dataframe()` | Convert to pandas DataFrame with metadata | | `to_llm_context()` | Generate AI-ready structured context | | `iter_hierarchy()` | Iterate with depth and parent information | | `iter_with_values()` | Iterate only items with values | | `find_item(concept)` | Find specific item by concept or label | | `to_dict()` | Convert to nested dictionary structure | | `to_flat_list()` | Convert to flat list for web APIs | | `get_period_comparison()` | Get period-over-period analysis | ### EntityFacts Enhanced Methods | Method | Description | | --- | --- | | `to_llm_context()` | Comprehensive AI context with focus areas | | `to_agent_tools()` | Export as MCP-compatible agent tools | | `calculate_ratios()` | Financial ratio calculations | | `peer_comparison()` | Compare with peer companies | | `detect_anomalies()` | Identify unusual patterns | ### New Parameters | Parameter | Methods | Description | | --- | --- | --- | | `concise_format` | All statement methods | Value display format control | | `include_metadata` | `to_llm_context()` | Include data quality metrics | | `flatten_values` | `to_llm_context()` | Flatten multi-period values | | `focus_areas` | `to_llm_context()` | Emphasize specific analysis areas | For complete API documentation of the underlying EntityFacts class and query interface, see the [EntityFacts API Reference](https://edgartools.readthedocs.io/en/stable/api/entity-facts-reference/) . * * * _The enhanced Company Facts API is part of EdgarTools' comprehensive SEC data platform, now with AI integration, web rendering support, and professional formatting. For more information, visit the [EdgarTools Documentation](https://edgartools.dev/) ._ Back to top --- # Multi-Period Analysis - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/multi-year-financial-data-api/#multi-year-financial-analysis-compare-sec-data-across-periods) Multi-Year Financial Analysis: Compare SEC Data Across Periods ============================================================== Learn how to access multiple years of financial statements and serve them through FastAPI endpoints for web applications and data analysis. Overview -------- This guide demonstrates how to retrieve historical financial data (income statement, balance sheet, and cash flow) for multiple years and expose it through professional FastAPI endpoints. Perfect for building financial dashboards, analysis tools, or data APIs. Quick Start ----------- `from edgar import Company from fastapi import FastAPI, HTTPException from typing import List, Dict, Any import pandas as pd app = FastAPI(title="Financial Data API") # Get multi-year financial data company = Company('AAPL') # Income statement - 5 years of annual data income_stmt = company.income_statement(periods=5, annual=True) # Balance sheet - 5 years of annual data balance_sheet = company.balance_sheet(periods=5, annual=True) # Cash flow - 5 years of annual data cash_flow = company.cashflow_statement(periods=5, annual=True) print(f"Retrieved {len(income_stmt.periods)} periods of data")` See it live on edgar.tools The code above builds multi-year financials with a FastAPI endpoint. **edgar.tools** already serves the same data — multi-period income statements, balance sheets, and cash flows for any company, with export to Excel, PDF, or CSV. * **[See Apple's multi-year financials →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=multi-year) ** * **[Browse revenue disclosures across all filing years →](https://app.edgar.tools/disclosures/revenue?utm_source=edgartools-docs&utm_medium=see-live&utm_content=multi-year) ** Also available via REST API — 20+ endpoints including multi-period financial data. [API docs →](https://app.edgar.tools/docs?utm_source=edgartools-docs&utm_medium=see-live&utm_content=multi-year) Core Data Retrieval Patterns ---------------------------- ### Multi-Year Annual Statements `def get_multi_year_financials(ticker: str, years: int = 5): """Get multiple years of financial statements""" company = Company(ticker) if not company.facts: return None # Get annual data for specified years financial_data = { 'company': { 'name': company.name, 'ticker': ticker, 'shares_outstanding': company.shares_outstanding, 'public_float': company.public_float }, 'income_statement': company.income_statement(periods=years, annual=True), 'balance_sheet': company.balance_sheet(periods=years, annual=True), 'cash_flow': company.cashflow_statement(periods=years, annual=True) } return financial_data # Example usage aapl_data = get_multi_year_financials('AAPL', 7) print(f"Retrieved data for periods: {aapl_data['income_statement'].periods}")` ### Mixed Period Analysis `def get_comprehensive_data(ticker: str): """Get both annual and quarterly data for trend analysis""" company = Company(ticker) if not company.facts: return None return { 'annual': { 'income': company.income_statement(periods=5, annual=True), 'balance': company.balance_sheet(periods=5, annual=True), 'cashflow': company.cashflow_statement(periods=5, annual=True) }, 'quarterly': { 'income': company.income_statement(periods=12, annual=False), 'balance': company.balance_sheet(periods=12, annual=False), 'cashflow': company.cashflow_statement(periods=12, annual=False) } } # Get comprehensive view comprehensive = get_comprehensive_data('MSFT')` FastAPI Endpoint Implementation ------------------------------- ### Basic Financial Data Endpoints `from fastapi import FastAPI, HTTPException, Query from pydantic import BaseModel from typing import Optional, Dict, Any, List from edgar import Company import json app = FastAPI( title="Financial Data API", description="Access multi-year financial statements from SEC data", version="1.0.0" ) class FinancialResponse(BaseModel): company_info: Dict[str, Any] periods: List[str] data: Dict[str, Any] metadata: Dict[str, Any] @app.get("/financial/{ticker}/income", response_model=FinancialResponse) async def get_income_statement( ticker: str, periods: int = Query(4, description="Number of periods", ge=1, le=20), annual: bool = Query(True, description="Annual (True) or Quarterly (False)"), concise_format: bool = Query(False, description="Use concise formatting ($1.0B vs $1,000,000,000)") ): """Get income statement data for multiple periods""" try: company = Company(ticker.upper()) if not company.facts: raise HTTPException(status_code=404, detail=f"No financial data available for {ticker}") # Get income statement stmt = company.income_statement(periods=periods, annual=annual, concise_format=concise_format) if not stmt: raise HTTPException(status_code=404, detail=f"No income statement data for {ticker}") # Convert to API response format response_data = { 'company_info': { 'name': company.name, 'ticker': ticker.upper(), 'shares_outstanding': company.shares_outstanding, 'public_float': company.public_float }, 'periods': stmt.periods, 'data': _convert_statement_to_dict(stmt), 'metadata': { 'period_type': 'annual' if annual else 'quarterly', 'concise_format': concise_format, 'total_items': len(list(stmt.iter_with_values())) } } return FinancialResponse(**response_data) except Exception as e: raise HTTPException(status_code=500, detail=str(e)) @app.get("/financial/{ticker}/balance", response_model=FinancialResponse) async def get_balance_sheet( ticker: str, periods: int = Query(4, description="Number of periods", ge=1, le=20), annual: bool = Query(True, description="Annual (True) or Quarterly (False)"), concise_format: bool = Query(False, description="Use concise formatting") ): """Get balance sheet data for multiple periods""" try: company = Company(ticker.upper()) if not company.facts: raise HTTPException(status_code=404, detail=f"No financial data available for {ticker}") stmt = company.balance_sheet(periods=periods, annual=annual, concise_format=concise_format) if not stmt: raise HTTPException(status_code=404, detail=f"No balance sheet data for {ticker}") response_data = { 'company_info': { 'name': company.name, 'ticker': ticker.upper(), 'shares_outstanding': company.shares_outstanding, 'public_float': company.public_float }, 'periods': stmt.periods, 'data': _convert_statement_to_dict(stmt), 'metadata': { 'period_type': 'annual' if annual else 'quarterly', 'concise_format': concise_format, 'total_items': len(list(stmt.iter_with_values())) } } return FinancialResponse(**response_data) except Exception as e: raise HTTPException(status_code=500, detail=str(e)) @app.get("/financial/{ticker}/cashflow", response_model=FinancialResponse) async def get_cash_flow( ticker: str, periods: int = Query(4, description="Number of periods", ge=1, le=20), annual: bool = Query(True, description="Annual (True) or Quarterly (False)"), concise_format: bool = Query(False, description="Use concise formatting") ): """Get cash flow statement data for multiple periods""" try: company = Company(ticker.upper()) if not company.facts: raise HTTPException(status_code=404, detail=f"No financial data available for {ticker}") stmt = company.cashflow_statement(periods=periods, annual=annual, concise_format=concise_format) if not stmt: raise HTTPException(status_code=404, detail=f"No cash flow data for {ticker}") response_data = { 'company_info': { 'name': company.name, 'ticker': ticker.upper(), 'shares_outstanding': company.shares_outstanding, 'public_float': company.public_float }, 'periods': stmt.periods, 'data': _convert_statement_to_dict(stmt), 'metadata': { 'period_type': 'annual' if annual else 'quarterly', 'concise_format': concise_format, 'total_items': len(list(stmt.iter_with_values())) } } return FinancialResponse(**response_data) except Exception as e: raise HTTPException(status_code=500, detail=str(e)) def _convert_statement_to_dict(stmt): """Convert statement to API-friendly dictionary format""" data = {} for item in stmt.iter_with_values(): # Create item data with all periods item_data = { 'label': item.label, 'concept': item.concept, 'values': {}, 'is_total': getattr(item, 'is_total', False), 'depth': getattr(item, 'depth', 0) } # Add values for each period for period in stmt.periods: value = item.values.get(period) if value is not None: item_data['values'][period] = { 'raw_value': value, 'display_value': item.get_display_value(period) } data[item.concept] = item_data return data` ### Comprehensive Multi-Statement Endpoint `class ComprehensiveFinancialResponse(BaseModel): company_info: Dict[str, Any] periods: List[str] income_statement: Dict[str, Any] balance_sheet: Dict[str, Any] cash_flow: Dict[str, Any] key_metrics: Dict[str, Any] metadata: Dict[str, Any] @app.get("/financial/{ticker}/comprehensive", response_model=ComprehensiveFinancialResponse) async def get_comprehensive_financials( ticker: str, periods: int = Query(5, description="Number of periods", ge=1, le=10), annual: bool = Query(True, description="Annual (True) or Quarterly (False)"), concise_format: bool = Query(False, description="Use concise formatting"), include_ratios: bool = Query(True, description="Calculate financial ratios") ): """Get comprehensive financial data including all statements and key metrics""" try: company = Company(ticker.upper()) if not company.facts: raise HTTPException(status_code=404, detail=f"No financial data available for {ticker}") # Get all three statements income_stmt = company.income_statement(periods=periods, annual=annual, concise_format=concise_format) balance_sheet = company.balance_sheet(periods=periods, annual=annual, concise_format=concise_format) cash_flow = company.cashflow_statement(periods=periods, annual=annual, concise_format=concise_format) # Get periods from the first available statement available_periods = [] if income_stmt: available_periods = income_stmt.periods elif balance_sheet: available_periods = balance_sheet.periods elif cash_flow: available_periods = cash_flow.periods if not available_periods: raise HTTPException(status_code=404, detail=f"No financial statement data available for {ticker}") # Calculate key metrics if requested key_metrics = {} if include_ratios and income_stmt and balance_sheet: key_metrics = _calculate_financial_ratios(income_stmt, balance_sheet, cash_flow) response_data = { 'company_info': { 'name': company.name, 'ticker': ticker.upper(), 'shares_outstanding': company.shares_outstanding, 'public_float': company.public_float }, 'periods': available_periods, 'income_statement': _convert_statement_to_dict(income_stmt) if income_stmt else {}, 'balance_sheet': _convert_statement_to_dict(balance_sheet) if balance_sheet else {}, 'cash_flow': _convert_statement_to_dict(cash_flow) if cash_flow else {}, 'key_metrics': key_metrics, 'metadata': { 'period_type': 'annual' if annual else 'quarterly', 'concise_format': concise_format, 'statements_available': { 'income_statement': income_stmt is not None, 'balance_sheet': balance_sheet is not None, 'cash_flow': cash_flow is not None } } } return ComprehensiveFinancialResponse(**response_data) except Exception as e: raise HTTPException(status_code=500, detail=str(e)) def _calculate_financial_ratios(income_stmt, balance_sheet, cash_flow): """Calculate key financial ratios from statements""" ratios = {} try: # Get latest period if not income_stmt.periods: return ratios latest_period = income_stmt.periods[0] # Find key items revenue_item = income_stmt.find_item('Revenue') net_income_item = income_stmt.find_item('Net Income') total_assets_item = balance_sheet.find_item('Assets') if balance_sheet else None total_equity_item = balance_sheet.find_item('Equity') if balance_sheet else None # Calculate ratios for latest period if revenue_item and net_income_item: revenue = revenue_item.values.get(latest_period) net_income = net_income_item.values.get(latest_period) if revenue and net_income and revenue != 0: ratios[f'profit_margin_{latest_period.lower().replace(" ", "_")}'] = net_income / revenue if net_income_item and total_assets_item: net_income = net_income_item.values.get(latest_period) total_assets = total_assets_item.values.get(latest_period) if net_income and total_assets and total_assets != 0: ratios[f'roa_{latest_period.lower().replace(" ", "_")}'] = net_income / total_assets if net_income_item and total_equity_item: net_income = net_income_item.values.get(latest_period) total_equity = total_equity_item.values.get(latest_period) if net_income and total_equity and total_equity != 0: ratios[f'roe_{latest_period.lower().replace(" ", "_")}'] = net_income / total_equity except Exception as e: # Log error but don't fail the request print(f"Error calculating ratios: {e}") return ratios` ### Historical Trend Analysis Endpoints `from datetime import datetime, timedelta @app.get("/financial/{ticker}/trends") async def get_financial_trends( ticker: str, metric: str = Query(..., description="Metric to analyze (revenue, net_income, assets)"), years: int = Query(5, description="Number of years", ge=2, le=10) ): """Get historical trends for specific financial metrics""" try: company = Company(ticker.upper()) if not company.facts: raise HTTPException(status_code=404, detail=f"No financial data available for {ticker}") # Determine which statement to use based on metric if metric.lower() in ['revenue', 'net_income', 'operating_income']: stmt = company.income_statement(periods=years, annual=True) elif metric.lower() in ['assets', 'liabilities', 'equity']: stmt = company.balance_sheet(periods=years, annual=True) elif metric.lower() in ['operating_cash_flow', 'free_cash_flow']: stmt = company.cashflow_statement(periods=years, annual=True) else: raise HTTPException(status_code=400, detail=f"Unknown metric: {metric}") if not stmt: raise HTTPException(status_code=404, detail=f"No data available for metric: {metric}") # Find the requested metric metric_item = stmt.find_item(metric) if not metric_item: # Try alternative names alt_names = { 'revenue': ['Revenues', 'Total Revenue', 'Net Sales'], 'net_income': ['Net Income', 'Net Earnings', 'Net Income (Loss)'], 'assets': ['Total Assets', 'Assets'], 'equity': ['Total Equity', 'Stockholders Equity', 'Total Stockholders Equity'] } for alt_name in alt_names.get(metric.lower(), []): metric_item = stmt.find_item(alt_name) if metric_item: break if not metric_item: raise HTTPException(status_code=404, detail=f"Metric '{metric}' not found in financial statements") # Calculate trends trend_data = [] values = [] for period in reversed(stmt.periods): # Chronological order value = metric_item.values.get(period) if value is not None: trend_data.append({ 'period': period, 'value': value, 'display_value': metric_item.get_display_value(period) }) values.append(value) # Calculate growth rates growth_rates = [] if len(values) > 1: for i in range(1, len(values)): if values[i-1] != 0: growth = ((values[i] - values[i-1]) / abs(values[i-1])) * 100 growth_rates.append(growth) return { 'company': company.name, 'ticker': ticker.upper(), 'metric': metric, 'periods_analyzed': len(trend_data), 'trend_data': trend_data, 'analytics': { 'average_growth_rate': sum(growth_rates) / len(growth_rates) if growth_rates else None, 'total_growth': ((values[-1] - values[0]) / abs(values[0])) * 100 if len(values) > 1 and values[0] != 0 else None, 'compound_annual_growth_rate': (((values[-1] / values[0]) ** (1/(len(values)-1))) - 1) * 100 if len(values) > 1 and values[0] > 0 else None } } except Exception as e: raise HTTPException(status_code=500, detail=str(e))` ### Bulk Company Comparison Endpoint `@app.post("/financial/compare") async def compare_companies( tickers: List[str], periods: int = Query(3, description="Number of periods", ge=1, le=5), metrics: List[str] = Query(['revenue', 'net_income'], description="Metrics to compare") ): """Compare financial metrics across multiple companies""" try: comparison_data = [] for ticker in tickers: try: company = Company(ticker.upper()) if not company.facts: continue company_data = { 'ticker': ticker.upper(), 'name': company.name, 'metrics': {} } # Get statements based on requested metrics income_stmt = company.income_statement(periods=periods, annual=True) balance_sheet = company.balance_sheet(periods=periods, annual=True) for metric in metrics: if metric.lower() in ['revenue', 'net_income']: stmt = income_stmt else: stmt = balance_sheet if stmt: metric_item = stmt.find_item(metric) if metric_item: company_data['metrics'][metric] = { 'periods': stmt.periods, 'values': metric_item.values } comparison_data.append(company_data) except Exception as e: print(f"Error processing {ticker}: {e}") continue return { 'comparison': comparison_data, 'metadata': { 'companies_compared': len(comparison_data), 'periods_requested': periods, 'metrics_requested': metrics } } except Exception as e: raise HTTPException(status_code=500, detail=str(e))` Running the API Server ---------------------- `# Save the above code as financial_api.py and run: if __name__ == "__main__": import uvicorn uvicorn.run( "financial_api:app", host="0.0.0.0", port=8000, reload=True, title="Financial Data API" ) # Or run from command line: # uvicorn financial_api:app --reload --host 0.0.0.0 --port 8000` API Usage Examples ------------------ ### Get 7 Years of Income Statement Data `# Get comprehensive income statement data curl "http://localhost:8000/financial/AAPL/income?periods=7&annual=true&concise_format=false" # Get quarterly data for recent periods curl "http://localhost:8000/financial/AAPL/income?periods=12&annual=false&concise_format=true"` ### Get Complete Financial Profile `# Get all three statements plus ratios curl "http://localhost:8000/financial/AAPL/comprehensive?periods=5&include_ratios=true"` ### Analyze Revenue Trends `# Get 10-year revenue trend analysis curl "http://localhost:8000/financial/AAPL/trends?metric=revenue&years=10"` ### Compare Multiple Companies `# Compare revenue and profits across companies curl -X POST "http://localhost:8000/financial/compare" \ -H "Content-Type: application/json" \ -d '{"tickers": ["AAPL", "MSFT", "GOOGL"], "periods": 5, "metrics": ["revenue", "net_income"]}'` Client Integration Examples --------------------------- ### Python Client `import requests import pandas as pd class FinancialDataClient: def __init__(self, base_url="http://localhost:8000"): self.base_url = base_url def get_income_statement(self, ticker, periods=5, annual=True): """Get income statement data""" response = requests.get( f"{self.base_url}/financial/{ticker}/income", params={'periods': periods, 'annual': annual} ) return response.json() if response.status_code == 200 else None def get_comprehensive_data(self, ticker, periods=5): """Get all financial statements""" response = requests.get( f"{self.base_url}/financial/{ticker}/comprehensive", params={'periods': periods, 'include_ratios': True} ) return response.json() if response.status_code == 200 else None def get_trends(self, ticker, metric, years=5): """Get trend analysis""" response = requests.get( f"{self.base_url}/financial/{ticker}/trends", params={'metric': metric, 'years': years} ) return response.json() if response.status_code == 200 else None # Usage client = FinancialDataClient() # Get Apple's financial data aapl_data = client.get_comprehensive_data('AAPL', periods=7) print(f"Retrieved data for periods: {aapl_data['periods']}") # Get revenue trends revenue_trends = client.get_trends('AAPL', 'revenue', years=10) print(f"10-year revenue CAGR: {revenue_trends['analytics']['compound_annual_growth_rate']:.1f}%")` ### JavaScript/Node.js Client ``class FinancialDataClient { constructor(baseUrl = 'http://localhost:8000') { this.baseUrl = baseUrl; } async getIncomeStatement(ticker, periods = 5, annual = true) { const response = await fetch( `${this.baseUrl}/financial/${ticker}/income?periods=${periods}&annual=${annual}` ); return response.ok ? response.json() : null; } async getComprehensiveData(ticker, periods = 5) { const response = await fetch( `${this.baseUrl}/financial/${ticker}/comprehensive?periods=${periods}&include_ratios=true` ); return response.ok ? response.json() : null; } async compareCompanies(tickers, periods = 3, metrics = ['revenue', 'net_income']) { const response = await fetch(`${this.baseUrl}/financial/compare`, { method: 'POST', headers: {'Content-Type': 'application/json'}, body: JSON.stringify({tickers, periods, metrics}) }); return response.ok ? response.json() : null; } } // Usage const client = new FinancialDataClient(); // Get multi-year data const msftData = await client.getComprehensiveData('MSFT', 7); console.log(`Retrieved data for periods:`, msftData.periods); // Compare tech giants const comparison = await client.compareCompanies(['AAPL', 'MSFT', 'GOOGL', 'AMZN']); console.log('Comparison data:', comparison);`` Advanced Use Cases ------------------ ### Building a Financial Dashboard `# dashboard.py - Streamlit financial dashboard import streamlit as st import requests import plotly.graph_objects as go import plotly.express as px import pandas as pd st.title("Multi-Year Financial Analysis Dashboard") # Input controls ticker = st.text_input("Company Ticker", value="AAPL") years = st.slider("Years of Data", min_value=2, max_value=10, value=5) if st.button("Analyze"): # Get data from our API client = FinancialDataClient() data = client.get_comprehensive_data(ticker, periods=years) if data: st.subheader(f"{data['company_info']['name']} ({ticker})") # Revenue trend chart income_data = data['income_statement'] revenue_concept = next((k for k in income_data.keys() if 'revenue' in k.lower()), None) if revenue_concept: revenue_item = income_data[revenue_concept] periods = data['periods'] values = [revenue_item['values'][p]['raw_value'] for p in periods if p in revenue_item['values']] fig = go.Figure() fig.add_trace(go.Scatter(x=periods, y=values, mode='lines+markers', name='Revenue')) fig.update_layout(title='Revenue Trend', xaxis_title='Period', yaxis_title='Revenue ($)') st.plotly_chart(fig) # Key metrics if data['key_metrics']: st.subheader("Key Financial Ratios") for metric, value in data['key_metrics'].items(): if isinstance(value, (int, float)): st.metric(metric.replace('_', ' ').title(), f"{value:.2%}" if 'margin' in metric or 'ro' in metric else f"{value:.2f}") # Run with: streamlit run dashboard.py` ### Data Export and Analysis `def export_financial_data_to_excel(tickers, periods=5, filename="financial_data.xlsx"): """Export multi-company financial data to Excel""" import pandas as pd from openpyxl import Workbook from openpyxl.utils.dataframe import dataframe_to_rows client = FinancialDataClient() with pd.ExcelWriter(filename, engine='openpyxl') as writer: for ticker in tickers: data = client.get_comprehensive_data(ticker, periods) if data: # Income Statement income_df = _convert_api_data_to_dataframe(data['income_statement'], data['periods']) income_df.to_excel(writer, sheet_name=f'{ticker}_Income') # Balance Sheet balance_df = _convert_api_data_to_dataframe(data['balance_sheet'], data['periods']) balance_df.to_excel(writer, sheet_name=f'{ticker}_Balance') # Cash Flow cashflow_df = _convert_api_data_to_dataframe(data['cash_flow'], data['periods']) cashflow_df.to_excel(writer, sheet_name=f'{ticker}_CashFlow') print(f"Financial data exported to {filename}") def _convert_api_data_to_dataframe(statement_data, periods): """Convert API response to pandas DataFrame""" rows = [] for concept, item_data in statement_data.items(): row = {'concept': concept, 'label': item_data['label']} for period in periods: if period in item_data['values']: row[period] = item_data['values'][period]['raw_value'] rows.append(row) return pd.DataFrame(rows) # Export data for analysis export_financial_data_to_excel(['AAPL', 'MSFT', 'GOOGL', 'AMZN'], periods=7)` Best Practices -------------- ### Error Handling and Resilience `import logging from functools import wraps def handle_api_errors(f): """Decorator for consistent API error handling""" @wraps(f) async def wrapper(*args, **kwargs): try: return await f(*args, **kwargs) except Exception as e: logging.error(f"API error in {f.__name__}: {str(e)}") raise HTTPException(status_code=500, detail=f"Internal server error: {str(e)}") return wrapper @app.get("/financial/{ticker}/income") @handle_api_errors async def get_income_statement_safe(ticker: str, periods: int = 4): # Implementation with automatic error handling pass` ### Caching for Performance `from functools import lru_cache from typing import Optional import time class CachedFinancialAPI: def __init__(self): self._cache = {} self._cache_ttl = 3600 # 1 hour def _get_cache_key(self, ticker: str, statement_type: str, periods: int, annual: bool): return f"{ticker}_{statement_type}_{periods}_{annual}" def _is_cache_valid(self, timestamp: float) -> bool: return time.time() - timestamp < self._cache_ttl @lru_cache(maxsize=100) def get_cached_company(self, ticker: str): """Cache Company objects to avoid repeated API calls""" return Company(ticker) def get_financial_data(self, ticker: str, statement_type: str, periods: int, annual: bool): cache_key = self._get_cache_key(ticker, statement_type, periods, annual) # Check cache if cache_key in self._cache: data, timestamp = self._cache[cache_key] if self._is_cache_valid(timestamp): return data # Fetch new data company = self.get_cached_company(ticker) if statement_type == 'income': data = company.income_statement(periods=periods, annual=annual) elif statement_type == 'balance': data = company.balance_sheet(periods=periods, annual=annual) elif statement_type == 'cashflow': data = company.cashflow_statement(periods=periods, annual=annual) # Cache the result self._cache[cache_key] = (data, time.time()) return data # Use cached API in endpoints cached_api = CachedFinancialAPI() @app.get("/financial/{ticker}/income") async def get_cached_income_statement(ticker: str, periods: int = 4, annual: bool = True): data = cached_api.get_financial_data(ticker, 'income', periods, annual) # ... rest of endpoint logic` Performance Optimization ------------------------ ### Batch Processing Multiple Companies `import asyncio import aiohttp from concurrent.futures import ThreadPoolExecutor async def process_companies_batch(tickers: List[str], periods: int = 5): """Process multiple companies in parallel""" def get_company_data(ticker): try: company = Company(ticker) if company.facts: return { 'ticker': ticker, 'income': company.income_statement(periods=periods, annual=True), 'balance': company.balance_sheet(periods=periods, annual=True), 'cashflow': company.cashflow_statement(periods=periods, annual=True) } except Exception as e: print(f"Error processing {ticker}: {e}") return None # Use ThreadPoolExecutor for I/O bound operations with ThreadPoolExecutor(max_workers=10) as executor: loop = asyncio.get_event_loop() tasks = [ loop.run_in_executor(executor, get_company_data, ticker) for ticker in tickers ] results = await asyncio.gather(*tasks) return [r for r in results if r is not None] @app.post("/financial/batch") async def process_batch(tickers: List[str], periods: int = Query(5, le=10)): """Process multiple companies in parallel""" results = await process_companies_batch(tickers, periods) return { 'processed': len(results), 'requested': len(tickers), 'data': results }` Deployment Considerations ------------------------- ### Production Setup `# production_config.py from fastapi import FastAPI from fastapi.middleware.cors import CORSMiddleware from fastapi.middleware.gzip import GZipMiddleware import logging def create_production_app(): app = FastAPI( title="Financial Data API", description="Production financial data service", version="1.0.0", docs_url="/docs" if settings.DEBUG else None ) # Add middleware app.add_middleware(GZipMiddleware, minimum_size=1000) app.add_middleware( CORSMiddleware, allow_origins=["https://yourdomain.com"], allow_credentials=True, allow_methods=["GET", "POST"], allow_headers=["*"], ) # Configure logging logging.basicConfig(level=logging.INFO) return app app = create_production_app() # Add rate limiting from slowapi import Limiter, _rate_limit_exceeded_handler from slowapi.util import get_remote_address from slowapi.errors import RateLimitExceeded limiter = Limiter(key_func=get_remote_address) app.state.limiter = limiter app.add_exception_handler(RateLimitExceeded, _rate_limit_exceeded_handler) @app.get("/financial/{ticker}/income") @limiter.limit("10/minute") async def rate_limited_income_statement(request: Request, ticker: str): # Rate-limited endpoint implementation pass` Testing ------- ### Unit Tests for API Endpoints `# test_financial_api.py import pytest from fastapi.testclient import TestClient from financial_api import app client = TestClient(app) def test_get_income_statement(): """Test income statement endpoint""" response = client.get("/financial/AAPL/income?periods=3&annual=true") assert response.status_code == 200 data = response.json() assert data['company_info']['ticker'] == 'AAPL' assert len(data['periods']) <= 3 assert 'income_statement' in data or 'data' in data def test_comprehensive_endpoint(): """Test comprehensive financial data endpoint""" response = client.get("/financial/MSFT/comprehensive?periods=2&include_ratios=true") assert response.status_code == 200 data = response.json() assert 'income_statement' in data assert 'balance_sheet' in data assert 'cash_flow' in data assert 'key_metrics' in data def test_trends_analysis(): """Test trends endpoint""" response = client.get("/financial/GOOGL/trends?metric=revenue&years=5") assert response.status_code == 200 data = response.json() assert 'trend_data' in data assert 'analytics' in data assert data['metric'] == 'revenue' def test_company_comparison(): """Test company comparison endpoint""" payload = { "tickers": ["AAPL", "MSFT"], "periods": 2, "metrics": ["revenue"] } response = client.post("/financial/compare", json=payload) assert response.status_code == 200 data = response.json() assert len(data['comparison']) <= 2 def test_invalid_ticker(): """Test handling of invalid ticker""" response = client.get("/financial/INVALIDTICKER/income") assert response.status_code == 404 # Run with: pytest test_financial_api.py -v` This comprehensive guide provides everything needed to build a production-ready FastAPI service for accessing multi-year financial statement data using EdgarTools. The implementation includes error handling, caching, rate limiting, and extensive examples for building financial analysis applications. \] Back to top --- # Multi-Period Analysis - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/multi-year-financial-data-api/#multi-year-financial-analysis-compare-sec-data-across-periods) Multi-Year Financial Analysis: Compare SEC Data Across Periods ============================================================== Learn how to access multiple years of financial statements and serve them through FastAPI endpoints for web applications and data analysis. Overview -------- This guide demonstrates how to retrieve historical financial data (income statement, balance sheet, and cash flow) for multiple years and expose it through professional FastAPI endpoints. Perfect for building financial dashboards, analysis tools, or data APIs. Quick Start ----------- `from edgar import Company from fastapi import FastAPI, HTTPException from typing import List, Dict, Any import pandas as pd app = FastAPI(title="Financial Data API") # Get multi-year financial data company = Company('AAPL') # Income statement - 5 years of annual data income_stmt = company.income_statement(periods=5, annual=True) # Balance sheet - 5 years of annual data balance_sheet = company.balance_sheet(periods=5, annual=True) # Cash flow - 5 years of annual data cash_flow = company.cashflow_statement(periods=5, annual=True) print(f"Retrieved {len(income_stmt.periods)} periods of data")` See it live on edgar.tools The code above builds multi-year financials with a FastAPI endpoint. **edgar.tools** already serves the same data — multi-period income statements, balance sheets, and cash flows for any company, with export to Excel, PDF, or CSV. * **[See Apple's multi-year financials →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=multi-year) ** * **[Browse revenue disclosures across all filing years →](https://app.edgar.tools/disclosures/revenue?utm_source=edgartools-docs&utm_medium=see-live&utm_content=multi-year) ** Also available via REST API — 20+ endpoints including multi-period financial data. [API docs →](https://app.edgar.tools/docs?utm_source=edgartools-docs&utm_medium=see-live&utm_content=multi-year) Core Data Retrieval Patterns ---------------------------- ### Multi-Year Annual Statements `def get_multi_year_financials(ticker: str, years: int = 5): """Get multiple years of financial statements""" company = Company(ticker) if not company.facts: return None # Get annual data for specified years financial_data = { 'company': { 'name': company.name, 'ticker': ticker, 'shares_outstanding': company.shares_outstanding, 'public_float': company.public_float }, 'income_statement': company.income_statement(periods=years, annual=True), 'balance_sheet': company.balance_sheet(periods=years, annual=True), 'cash_flow': company.cashflow_statement(periods=years, annual=True) } return financial_data # Example usage aapl_data = get_multi_year_financials('AAPL', 7) print(f"Retrieved data for periods: {aapl_data['income_statement'].periods}")` ### Mixed Period Analysis `def get_comprehensive_data(ticker: str): """Get both annual and quarterly data for trend analysis""" company = Company(ticker) if not company.facts: return None return { 'annual': { 'income': company.income_statement(periods=5, annual=True), 'balance': company.balance_sheet(periods=5, annual=True), 'cashflow': company.cashflow_statement(periods=5, annual=True) }, 'quarterly': { 'income': company.income_statement(periods=12, annual=False), 'balance': company.balance_sheet(periods=12, annual=False), 'cashflow': company.cashflow_statement(periods=12, annual=False) } } # Get comprehensive view comprehensive = get_comprehensive_data('MSFT')` FastAPI Endpoint Implementation ------------------------------- ### Basic Financial Data Endpoints `from fastapi import FastAPI, HTTPException, Query from pydantic import BaseModel from typing import Optional, Dict, Any, List from edgar import Company import json app = FastAPI( title="Financial Data API", description="Access multi-year financial statements from SEC data", version="1.0.0" ) class FinancialResponse(BaseModel): company_info: Dict[str, Any] periods: List[str] data: Dict[str, Any] metadata: Dict[str, Any] @app.get("/financial/{ticker}/income", response_model=FinancialResponse) async def get_income_statement( ticker: str, periods: int = Query(4, description="Number of periods", ge=1, le=20), annual: bool = Query(True, description="Annual (True) or Quarterly (False)"), concise_format: bool = Query(False, description="Use concise formatting ($1.0B vs $1,000,000,000)") ): """Get income statement data for multiple periods""" try: company = Company(ticker.upper()) if not company.facts: raise HTTPException(status_code=404, detail=f"No financial data available for {ticker}") # Get income statement stmt = company.income_statement(periods=periods, annual=annual, concise_format=concise_format) if not stmt: raise HTTPException(status_code=404, detail=f"No income statement data for {ticker}") # Convert to API response format response_data = { 'company_info': { 'name': company.name, 'ticker': ticker.upper(), 'shares_outstanding': company.shares_outstanding, 'public_float': company.public_float }, 'periods': stmt.periods, 'data': _convert_statement_to_dict(stmt), 'metadata': { 'period_type': 'annual' if annual else 'quarterly', 'concise_format': concise_format, 'total_items': len(list(stmt.iter_with_values())) } } return FinancialResponse(**response_data) except Exception as e: raise HTTPException(status_code=500, detail=str(e)) @app.get("/financial/{ticker}/balance", response_model=FinancialResponse) async def get_balance_sheet( ticker: str, periods: int = Query(4, description="Number of periods", ge=1, le=20), annual: bool = Query(True, description="Annual (True) or Quarterly (False)"), concise_format: bool = Query(False, description="Use concise formatting") ): """Get balance sheet data for multiple periods""" try: company = Company(ticker.upper()) if not company.facts: raise HTTPException(status_code=404, detail=f"No financial data available for {ticker}") stmt = company.balance_sheet(periods=periods, annual=annual, concise_format=concise_format) if not stmt: raise HTTPException(status_code=404, detail=f"No balance sheet data for {ticker}") response_data = { 'company_info': { 'name': company.name, 'ticker': ticker.upper(), 'shares_outstanding': company.shares_outstanding, 'public_float': company.public_float }, 'periods': stmt.periods, 'data': _convert_statement_to_dict(stmt), 'metadata': { 'period_type': 'annual' if annual else 'quarterly', 'concise_format': concise_format, 'total_items': len(list(stmt.iter_with_values())) } } return FinancialResponse(**response_data) except Exception as e: raise HTTPException(status_code=500, detail=str(e)) @app.get("/financial/{ticker}/cashflow", response_model=FinancialResponse) async def get_cash_flow( ticker: str, periods: int = Query(4, description="Number of periods", ge=1, le=20), annual: bool = Query(True, description="Annual (True) or Quarterly (False)"), concise_format: bool = Query(False, description="Use concise formatting") ): """Get cash flow statement data for multiple periods""" try: company = Company(ticker.upper()) if not company.facts: raise HTTPException(status_code=404, detail=f"No financial data available for {ticker}") stmt = company.cashflow_statement(periods=periods, annual=annual, concise_format=concise_format) if not stmt: raise HTTPException(status_code=404, detail=f"No cash flow data for {ticker}") response_data = { 'company_info': { 'name': company.name, 'ticker': ticker.upper(), 'shares_outstanding': company.shares_outstanding, 'public_float': company.public_float }, 'periods': stmt.periods, 'data': _convert_statement_to_dict(stmt), 'metadata': { 'period_type': 'annual' if annual else 'quarterly', 'concise_format': concise_format, 'total_items': len(list(stmt.iter_with_values())) } } return FinancialResponse(**response_data) except Exception as e: raise HTTPException(status_code=500, detail=str(e)) def _convert_statement_to_dict(stmt): """Convert statement to API-friendly dictionary format""" data = {} for item in stmt.iter_with_values(): # Create item data with all periods item_data = { 'label': item.label, 'concept': item.concept, 'values': {}, 'is_total': getattr(item, 'is_total', False), 'depth': getattr(item, 'depth', 0) } # Add values for each period for period in stmt.periods: value = item.values.get(period) if value is not None: item_data['values'][period] = { 'raw_value': value, 'display_value': item.get_display_value(period) } data[item.concept] = item_data return data` ### Comprehensive Multi-Statement Endpoint `class ComprehensiveFinancialResponse(BaseModel): company_info: Dict[str, Any] periods: List[str] income_statement: Dict[str, Any] balance_sheet: Dict[str, Any] cash_flow: Dict[str, Any] key_metrics: Dict[str, Any] metadata: Dict[str, Any] @app.get("/financial/{ticker}/comprehensive", response_model=ComprehensiveFinancialResponse) async def get_comprehensive_financials( ticker: str, periods: int = Query(5, description="Number of periods", ge=1, le=10), annual: bool = Query(True, description="Annual (True) or Quarterly (False)"), concise_format: bool = Query(False, description="Use concise formatting"), include_ratios: bool = Query(True, description="Calculate financial ratios") ): """Get comprehensive financial data including all statements and key metrics""" try: company = Company(ticker.upper()) if not company.facts: raise HTTPException(status_code=404, detail=f"No financial data available for {ticker}") # Get all three statements income_stmt = company.income_statement(periods=periods, annual=annual, concise_format=concise_format) balance_sheet = company.balance_sheet(periods=periods, annual=annual, concise_format=concise_format) cash_flow = company.cashflow_statement(periods=periods, annual=annual, concise_format=concise_format) # Get periods from the first available statement available_periods = [] if income_stmt: available_periods = income_stmt.periods elif balance_sheet: available_periods = balance_sheet.periods elif cash_flow: available_periods = cash_flow.periods if not available_periods: raise HTTPException(status_code=404, detail=f"No financial statement data available for {ticker}") # Calculate key metrics if requested key_metrics = {} if include_ratios and income_stmt and balance_sheet: key_metrics = _calculate_financial_ratios(income_stmt, balance_sheet, cash_flow) response_data = { 'company_info': { 'name': company.name, 'ticker': ticker.upper(), 'shares_outstanding': company.shares_outstanding, 'public_float': company.public_float }, 'periods': available_periods, 'income_statement': _convert_statement_to_dict(income_stmt) if income_stmt else {}, 'balance_sheet': _convert_statement_to_dict(balance_sheet) if balance_sheet else {}, 'cash_flow': _convert_statement_to_dict(cash_flow) if cash_flow else {}, 'key_metrics': key_metrics, 'metadata': { 'period_type': 'annual' if annual else 'quarterly', 'concise_format': concise_format, 'statements_available': { 'income_statement': income_stmt is not None, 'balance_sheet': balance_sheet is not None, 'cash_flow': cash_flow is not None } } } return ComprehensiveFinancialResponse(**response_data) except Exception as e: raise HTTPException(status_code=500, detail=str(e)) def _calculate_financial_ratios(income_stmt, balance_sheet, cash_flow): """Calculate key financial ratios from statements""" ratios = {} try: # Get latest period if not income_stmt.periods: return ratios latest_period = income_stmt.periods[0] # Find key items revenue_item = income_stmt.find_item('Revenue') net_income_item = income_stmt.find_item('Net Income') total_assets_item = balance_sheet.find_item('Assets') if balance_sheet else None total_equity_item = balance_sheet.find_item('Equity') if balance_sheet else None # Calculate ratios for latest period if revenue_item and net_income_item: revenue = revenue_item.values.get(latest_period) net_income = net_income_item.values.get(latest_period) if revenue and net_income and revenue != 0: ratios[f'profit_margin_{latest_period.lower().replace(" ", "_")}'] = net_income / revenue if net_income_item and total_assets_item: net_income = net_income_item.values.get(latest_period) total_assets = total_assets_item.values.get(latest_period) if net_income and total_assets and total_assets != 0: ratios[f'roa_{latest_period.lower().replace(" ", "_")}'] = net_income / total_assets if net_income_item and total_equity_item: net_income = net_income_item.values.get(latest_period) total_equity = total_equity_item.values.get(latest_period) if net_income and total_equity and total_equity != 0: ratios[f'roe_{latest_period.lower().replace(" ", "_")}'] = net_income / total_equity except Exception as e: # Log error but don't fail the request print(f"Error calculating ratios: {e}") return ratios` ### Historical Trend Analysis Endpoints `from datetime import datetime, timedelta @app.get("/financial/{ticker}/trends") async def get_financial_trends( ticker: str, metric: str = Query(..., description="Metric to analyze (revenue, net_income, assets)"), years: int = Query(5, description="Number of years", ge=2, le=10) ): """Get historical trends for specific financial metrics""" try: company = Company(ticker.upper()) if not company.facts: raise HTTPException(status_code=404, detail=f"No financial data available for {ticker}") # Determine which statement to use based on metric if metric.lower() in ['revenue', 'net_income', 'operating_income']: stmt = company.income_statement(periods=years, annual=True) elif metric.lower() in ['assets', 'liabilities', 'equity']: stmt = company.balance_sheet(periods=years, annual=True) elif metric.lower() in ['operating_cash_flow', 'free_cash_flow']: stmt = company.cashflow_statement(periods=years, annual=True) else: raise HTTPException(status_code=400, detail=f"Unknown metric: {metric}") if not stmt: raise HTTPException(status_code=404, detail=f"No data available for metric: {metric}") # Find the requested metric metric_item = stmt.find_item(metric) if not metric_item: # Try alternative names alt_names = { 'revenue': ['Revenues', 'Total Revenue', 'Net Sales'], 'net_income': ['Net Income', 'Net Earnings', 'Net Income (Loss)'], 'assets': ['Total Assets', 'Assets'], 'equity': ['Total Equity', 'Stockholders Equity', 'Total Stockholders Equity'] } for alt_name in alt_names.get(metric.lower(), []): metric_item = stmt.find_item(alt_name) if metric_item: break if not metric_item: raise HTTPException(status_code=404, detail=f"Metric '{metric}' not found in financial statements") # Calculate trends trend_data = [] values = [] for period in reversed(stmt.periods): # Chronological order value = metric_item.values.get(period) if value is not None: trend_data.append({ 'period': period, 'value': value, 'display_value': metric_item.get_display_value(period) }) values.append(value) # Calculate growth rates growth_rates = [] if len(values) > 1: for i in range(1, len(values)): if values[i-1] != 0: growth = ((values[i] - values[i-1]) / abs(values[i-1])) * 100 growth_rates.append(growth) return { 'company': company.name, 'ticker': ticker.upper(), 'metric': metric, 'periods_analyzed': len(trend_data), 'trend_data': trend_data, 'analytics': { 'average_growth_rate': sum(growth_rates) / len(growth_rates) if growth_rates else None, 'total_growth': ((values[-1] - values[0]) / abs(values[0])) * 100 if len(values) > 1 and values[0] != 0 else None, 'compound_annual_growth_rate': (((values[-1] / values[0]) ** (1/(len(values)-1))) - 1) * 100 if len(values) > 1 and values[0] > 0 else None } } except Exception as e: raise HTTPException(status_code=500, detail=str(e))` ### Bulk Company Comparison Endpoint `@app.post("/financial/compare") async def compare_companies( tickers: List[str], periods: int = Query(3, description="Number of periods", ge=1, le=5), metrics: List[str] = Query(['revenue', 'net_income'], description="Metrics to compare") ): """Compare financial metrics across multiple companies""" try: comparison_data = [] for ticker in tickers: try: company = Company(ticker.upper()) if not company.facts: continue company_data = { 'ticker': ticker.upper(), 'name': company.name, 'metrics': {} } # Get statements based on requested metrics income_stmt = company.income_statement(periods=periods, annual=True) balance_sheet = company.balance_sheet(periods=periods, annual=True) for metric in metrics: if metric.lower() in ['revenue', 'net_income']: stmt = income_stmt else: stmt = balance_sheet if stmt: metric_item = stmt.find_item(metric) if metric_item: company_data['metrics'][metric] = { 'periods': stmt.periods, 'values': metric_item.values } comparison_data.append(company_data) except Exception as e: print(f"Error processing {ticker}: {e}") continue return { 'comparison': comparison_data, 'metadata': { 'companies_compared': len(comparison_data), 'periods_requested': periods, 'metrics_requested': metrics } } except Exception as e: raise HTTPException(status_code=500, detail=str(e))` Running the API Server ---------------------- `# Save the above code as financial_api.py and run: if __name__ == "__main__": import uvicorn uvicorn.run( "financial_api:app", host="0.0.0.0", port=8000, reload=True, title="Financial Data API" ) # Or run from command line: # uvicorn financial_api:app --reload --host 0.0.0.0 --port 8000` API Usage Examples ------------------ ### Get 7 Years of Income Statement Data `# Get comprehensive income statement data curl "http://localhost:8000/financial/AAPL/income?periods=7&annual=true&concise_format=false" # Get quarterly data for recent periods curl "http://localhost:8000/financial/AAPL/income?periods=12&annual=false&concise_format=true"` ### Get Complete Financial Profile `# Get all three statements plus ratios curl "http://localhost:8000/financial/AAPL/comprehensive?periods=5&include_ratios=true"` ### Analyze Revenue Trends `# Get 10-year revenue trend analysis curl "http://localhost:8000/financial/AAPL/trends?metric=revenue&years=10"` ### Compare Multiple Companies `# Compare revenue and profits across companies curl -X POST "http://localhost:8000/financial/compare" \ -H "Content-Type: application/json" \ -d '{"tickers": ["AAPL", "MSFT", "GOOGL"], "periods": 5, "metrics": ["revenue", "net_income"]}'` Client Integration Examples --------------------------- ### Python Client `import requests import pandas as pd class FinancialDataClient: def __init__(self, base_url="http://localhost:8000"): self.base_url = base_url def get_income_statement(self, ticker, periods=5, annual=True): """Get income statement data""" response = requests.get( f"{self.base_url}/financial/{ticker}/income", params={'periods': periods, 'annual': annual} ) return response.json() if response.status_code == 200 else None def get_comprehensive_data(self, ticker, periods=5): """Get all financial statements""" response = requests.get( f"{self.base_url}/financial/{ticker}/comprehensive", params={'periods': periods, 'include_ratios': True} ) return response.json() if response.status_code == 200 else None def get_trends(self, ticker, metric, years=5): """Get trend analysis""" response = requests.get( f"{self.base_url}/financial/{ticker}/trends", params={'metric': metric, 'years': years} ) return response.json() if response.status_code == 200 else None # Usage client = FinancialDataClient() # Get Apple's financial data aapl_data = client.get_comprehensive_data('AAPL', periods=7) print(f"Retrieved data for periods: {aapl_data['periods']}") # Get revenue trends revenue_trends = client.get_trends('AAPL', 'revenue', years=10) print(f"10-year revenue CAGR: {revenue_trends['analytics']['compound_annual_growth_rate']:.1f}%")` ### JavaScript/Node.js Client ``class FinancialDataClient { constructor(baseUrl = 'http://localhost:8000') { this.baseUrl = baseUrl; } async getIncomeStatement(ticker, periods = 5, annual = true) { const response = await fetch( `${this.baseUrl}/financial/${ticker}/income?periods=${periods}&annual=${annual}` ); return response.ok ? response.json() : null; } async getComprehensiveData(ticker, periods = 5) { const response = await fetch( `${this.baseUrl}/financial/${ticker}/comprehensive?periods=${periods}&include_ratios=true` ); return response.ok ? response.json() : null; } async compareCompanies(tickers, periods = 3, metrics = ['revenue', 'net_income']) { const response = await fetch(`${this.baseUrl}/financial/compare`, { method: 'POST', headers: {'Content-Type': 'application/json'}, body: JSON.stringify({tickers, periods, metrics}) }); return response.ok ? response.json() : null; } } // Usage const client = new FinancialDataClient(); // Get multi-year data const msftData = await client.getComprehensiveData('MSFT', 7); console.log(`Retrieved data for periods:`, msftData.periods); // Compare tech giants const comparison = await client.compareCompanies(['AAPL', 'MSFT', 'GOOGL', 'AMZN']); console.log('Comparison data:', comparison);`` Advanced Use Cases ------------------ ### Building a Financial Dashboard `# dashboard.py - Streamlit financial dashboard import streamlit as st import requests import plotly.graph_objects as go import plotly.express as px import pandas as pd st.title("Multi-Year Financial Analysis Dashboard") # Input controls ticker = st.text_input("Company Ticker", value="AAPL") years = st.slider("Years of Data", min_value=2, max_value=10, value=5) if st.button("Analyze"): # Get data from our API client = FinancialDataClient() data = client.get_comprehensive_data(ticker, periods=years) if data: st.subheader(f"{data['company_info']['name']} ({ticker})") # Revenue trend chart income_data = data['income_statement'] revenue_concept = next((k for k in income_data.keys() if 'revenue' in k.lower()), None) if revenue_concept: revenue_item = income_data[revenue_concept] periods = data['periods'] values = [revenue_item['values'][p]['raw_value'] for p in periods if p in revenue_item['values']] fig = go.Figure() fig.add_trace(go.Scatter(x=periods, y=values, mode='lines+markers', name='Revenue')) fig.update_layout(title='Revenue Trend', xaxis_title='Period', yaxis_title='Revenue ($)') st.plotly_chart(fig) # Key metrics if data['key_metrics']: st.subheader("Key Financial Ratios") for metric, value in data['key_metrics'].items(): if isinstance(value, (int, float)): st.metric(metric.replace('_', ' ').title(), f"{value:.2%}" if 'margin' in metric or 'ro' in metric else f"{value:.2f}") # Run with: streamlit run dashboard.py` ### Data Export and Analysis `def export_financial_data_to_excel(tickers, periods=5, filename="financial_data.xlsx"): """Export multi-company financial data to Excel""" import pandas as pd from openpyxl import Workbook from openpyxl.utils.dataframe import dataframe_to_rows client = FinancialDataClient() with pd.ExcelWriter(filename, engine='openpyxl') as writer: for ticker in tickers: data = client.get_comprehensive_data(ticker, periods) if data: # Income Statement income_df = _convert_api_data_to_dataframe(data['income_statement'], data['periods']) income_df.to_excel(writer, sheet_name=f'{ticker}_Income') # Balance Sheet balance_df = _convert_api_data_to_dataframe(data['balance_sheet'], data['periods']) balance_df.to_excel(writer, sheet_name=f'{ticker}_Balance') # Cash Flow cashflow_df = _convert_api_data_to_dataframe(data['cash_flow'], data['periods']) cashflow_df.to_excel(writer, sheet_name=f'{ticker}_CashFlow') print(f"Financial data exported to {filename}") def _convert_api_data_to_dataframe(statement_data, periods): """Convert API response to pandas DataFrame""" rows = [] for concept, item_data in statement_data.items(): row = {'concept': concept, 'label': item_data['label']} for period in periods: if period in item_data['values']: row[period] = item_data['values'][period]['raw_value'] rows.append(row) return pd.DataFrame(rows) # Export data for analysis export_financial_data_to_excel(['AAPL', 'MSFT', 'GOOGL', 'AMZN'], periods=7)` Best Practices -------------- ### Error Handling and Resilience `import logging from functools import wraps def handle_api_errors(f): """Decorator for consistent API error handling""" @wraps(f) async def wrapper(*args, **kwargs): try: return await f(*args, **kwargs) except Exception as e: logging.error(f"API error in {f.__name__}: {str(e)}") raise HTTPException(status_code=500, detail=f"Internal server error: {str(e)}") return wrapper @app.get("/financial/{ticker}/income") @handle_api_errors async def get_income_statement_safe(ticker: str, periods: int = 4): # Implementation with automatic error handling pass` ### Caching for Performance `from functools import lru_cache from typing import Optional import time class CachedFinancialAPI: def __init__(self): self._cache = {} self._cache_ttl = 3600 # 1 hour def _get_cache_key(self, ticker: str, statement_type: str, periods: int, annual: bool): return f"{ticker}_{statement_type}_{periods}_{annual}" def _is_cache_valid(self, timestamp: float) -> bool: return time.time() - timestamp < self._cache_ttl @lru_cache(maxsize=100) def get_cached_company(self, ticker: str): """Cache Company objects to avoid repeated API calls""" return Company(ticker) def get_financial_data(self, ticker: str, statement_type: str, periods: int, annual: bool): cache_key = self._get_cache_key(ticker, statement_type, periods, annual) # Check cache if cache_key in self._cache: data, timestamp = self._cache[cache_key] if self._is_cache_valid(timestamp): return data # Fetch new data company = self.get_cached_company(ticker) if statement_type == 'income': data = company.income_statement(periods=periods, annual=annual) elif statement_type == 'balance': data = company.balance_sheet(periods=periods, annual=annual) elif statement_type == 'cashflow': data = company.cashflow_statement(periods=periods, annual=annual) # Cache the result self._cache[cache_key] = (data, time.time()) return data # Use cached API in endpoints cached_api = CachedFinancialAPI() @app.get("/financial/{ticker}/income") async def get_cached_income_statement(ticker: str, periods: int = 4, annual: bool = True): data = cached_api.get_financial_data(ticker, 'income', periods, annual) # ... rest of endpoint logic` Performance Optimization ------------------------ ### Batch Processing Multiple Companies `import asyncio import aiohttp from concurrent.futures import ThreadPoolExecutor async def process_companies_batch(tickers: List[str], periods: int = 5): """Process multiple companies in parallel""" def get_company_data(ticker): try: company = Company(ticker) if company.facts: return { 'ticker': ticker, 'income': company.income_statement(periods=periods, annual=True), 'balance': company.balance_sheet(periods=periods, annual=True), 'cashflow': company.cashflow_statement(periods=periods, annual=True) } except Exception as e: print(f"Error processing {ticker}: {e}") return None # Use ThreadPoolExecutor for I/O bound operations with ThreadPoolExecutor(max_workers=10) as executor: loop = asyncio.get_event_loop() tasks = [ loop.run_in_executor(executor, get_company_data, ticker) for ticker in tickers ] results = await asyncio.gather(*tasks) return [r for r in results if r is not None] @app.post("/financial/batch") async def process_batch(tickers: List[str], periods: int = Query(5, le=10)): """Process multiple companies in parallel""" results = await process_companies_batch(tickers, periods) return { 'processed': len(results), 'requested': len(tickers), 'data': results }` Deployment Considerations ------------------------- ### Production Setup `# production_config.py from fastapi import FastAPI from fastapi.middleware.cors import CORSMiddleware from fastapi.middleware.gzip import GZipMiddleware import logging def create_production_app(): app = FastAPI( title="Financial Data API", description="Production financial data service", version="1.0.0", docs_url="/docs" if settings.DEBUG else None ) # Add middleware app.add_middleware(GZipMiddleware, minimum_size=1000) app.add_middleware( CORSMiddleware, allow_origins=["https://yourdomain.com"], allow_credentials=True, allow_methods=["GET", "POST"], allow_headers=["*"], ) # Configure logging logging.basicConfig(level=logging.INFO) return app app = create_production_app() # Add rate limiting from slowapi import Limiter, _rate_limit_exceeded_handler from slowapi.util import get_remote_address from slowapi.errors import RateLimitExceeded limiter = Limiter(key_func=get_remote_address) app.state.limiter = limiter app.add_exception_handler(RateLimitExceeded, _rate_limit_exceeded_handler) @app.get("/financial/{ticker}/income") @limiter.limit("10/minute") async def rate_limited_income_statement(request: Request, ticker: str): # Rate-limited endpoint implementation pass` Testing ------- ### Unit Tests for API Endpoints `# test_financial_api.py import pytest from fastapi.testclient import TestClient from financial_api import app client = TestClient(app) def test_get_income_statement(): """Test income statement endpoint""" response = client.get("/financial/AAPL/income?periods=3&annual=true") assert response.status_code == 200 data = response.json() assert data['company_info']['ticker'] == 'AAPL' assert len(data['periods']) <= 3 assert 'income_statement' in data or 'data' in data def test_comprehensive_endpoint(): """Test comprehensive financial data endpoint""" response = client.get("/financial/MSFT/comprehensive?periods=2&include_ratios=true") assert response.status_code == 200 data = response.json() assert 'income_statement' in data assert 'balance_sheet' in data assert 'cash_flow' in data assert 'key_metrics' in data def test_trends_analysis(): """Test trends endpoint""" response = client.get("/financial/GOOGL/trends?metric=revenue&years=5") assert response.status_code == 200 data = response.json() assert 'trend_data' in data assert 'analytics' in data assert data['metric'] == 'revenue' def test_company_comparison(): """Test company comparison endpoint""" payload = { "tickers": ["AAPL", "MSFT"], "periods": 2, "metrics": ["revenue"] } response = client.post("/financial/compare", json=payload) assert response.status_code == 200 data = response.json() assert len(data['comparison']) <= 2 def test_invalid_ticker(): """Test handling of invalid ticker""" response = client.get("/financial/INVALIDTICKER/income") assert response.status_code == 404 # Run with: pytest test_financial_api.py -v` This comprehensive guide provides everything needed to build a production-ready FastAPI service for accessing multi-year financial statement data using EdgarTools. The implementation includes error handling, caching, rate limiting, and extensive examples for building financial analysis applications. \] Back to top --- # Current Events (8-K) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/eightk-filings/#8-k-current-reports-parse-sec-corporate-events-and-earnings-with-python) 8-K Current Reports: Parse SEC Corporate Events and Earnings with Python ======================================================================== Companies file 8-K current reports within four business days of material events -- acquisitions, executive changes, earnings releases, bankruptcy. EdgarTools parses these filings into structured Python objects so you can access the event items, press releases, and financial tables. `from edgar import * filing = get_filings(form="8-K").latest() eight_k = filing.obj() eight_k` ![8-K current report parsed with Python edgartools](https://edgartools.readthedocs.io/en/latest/images/eightk.webp) Three lines to get a parsed 8-K with company info, filing date, items disclosed, and exhibits. > **[See today's 8-K filings on edgar.tools — with AI-classified material event types →](https://app.edgar.tools/filings?utm_source=edgartools-docs&utm_medium=see-live&utm_content=eightk) > ** * * * Read Item Content ----------------- Every 8-K discloses one or more numbered items (1.01 through 9.01). The `items` property lists what was disclosed: `eight_k.items # ['Item 2.02', 'Item 9.01']` Access the text of any item by number: `# Works with or without "Item" prefix content = eight_k['2.02'] content = eight_k['Item 2.02'] print(content)` Common items: | Item | What it reports | | --- | --- | | **1.01** | Material agreements | | **2.02** | Earnings and financial condition | | **5.02** | Director or officer changes | | **8.01** | Other events | | **9.01** | Financial statements and exhibits | The complete mapping is in `eight_k.structure`. * * * Access Press Releases --------------------- Most 8-Ks attach press releases as EX-99 exhibits: `if eight_k.has_press_release: releases = eight_k.press_releases pr = releases[0] # Get content in different formats pr.text() # Plain text pr.html() # HTML pr.to_markdown() # Markdown pr.open() # Open in browser` Press releases are indexed by position. `press_releases[0]` is the first release, `press_releases[1]` is the second. * * * Extract Earnings and Financial Statements from 8-K Filings ---------------------------------------------------------- When a company reports quarterly earnings, the numbers first appear in an 8-K -- weeks before the formal 10-Q. The company files **Item 2.02** ("Results of Operations and Financial Condition") and attaches the earnings press release as an EX-99.1 exhibit. That exhibit contains HTML tables with income statements, balance sheets, and cash flows. EdgarTools detects this pattern and parses the tables automatically. `from edgar import Company aapl = Company("AAPL") filing = aapl.get_filings(form="8-K").latest() eight_k = filing.obj() eight_k.has_earnings # True if Item 2.02 + parseable EX-99.1 exhibit` ### Get Financial Statements `if eight_k.has_earnings: # Direct access to parsed statements eight_k.income_statement # FinancialTable or None eight_k.balance_sheet # FinancialTable or None eight_k.cash_flow_statement # FinancialTable or None # Safe accessors -- always return a DataFrame (empty if missing) df = eight_k.get_income_statement() df = eight_k.get_balance_sheet() df = eight_k.get_cash_flow_statement()` ### Work with the Data Each statement is a `FinancialTable` with the parsed data and detected scale: `income = eight_k.income_statement income.dataframe # Parsed data as DataFrame income.scaled_dataframe # Values multiplied by detected scale income.scale # Scale enum (UNITS, THOUSANDS, MILLIONS, BILLIONS) income.title # Table title if detected income.to_html() # HTML export for web apps income.to_json() # JSON export for APIs` Press releases often report values "in millions" or "in thousands." EdgarTools detects the scale from the document text. Use `scaled_dataframe` to get actual dollar amounts, or check `scale` to apply your own multiplier. ### Access All Parsed Tables The `earnings` property returns the full `EarningsRelease` object, which may contain more than just the three core statements: `earnings = eight_k.earnings earnings.financial_tables # All parsed financial tables earnings.segment_data # Business segment breakdown earnings.eps_reconciliation # GAAP to Non-GAAP EPS reconciliation earnings.guidance # Forward guidance table earnings.detected_scale # Document-level scale factor` Not every press release includes all of these -- `segment_data`, `eps_reconciliation`, and `guidance` return `None` when the company doesn't report them. * * * Work with Exhibits ------------------ All 8-K attachments (press releases, financial statements, material agreements): `exhibits = filing.exhibits for ex in exhibits: print(f"{ex.document_type}: {ex.description}") # Access specific exhibit ex_99 = exhibits[0] content = ex_99.download()` Exhibits are indexed by position. The `document_type` shows what kind of exhibit it is (EX-99.1, EX-10.1, etc.). * * * Common Analysis Patterns ------------------------ ### Find all earnings releases in a quarter `from edgar import get_filings filings = get_filings( form="8-K", date="2024-01-01:2024-03-31" ) for filing in filings[:20]: eight_k = filing.obj() if eight_k.has_earnings: print(f"{filing.company}: {filing.filing_date}")` ### Extract all financial tables `if eight_k.has_earnings: for table in eight_k.earnings.financial_tables: print(f"{table.statement_type.value}: {table.dataframe.shape}")` ### Check for specific events `# Director changes if 'Item 5.02' in eight_k.items: print(eight_k['5.02']) # Material agreements if 'Item 1.01' in eight_k.items: print(eight_k['1.01']) # Stock split announcements (Item 8.01 or 5.03) if 'Item 8.01' in eight_k.items or 'Item 5.03' in eight_k.items: content = eight_k.get('8.01') or eight_k.get('5.03') or '' if 'split' in content.lower(): print(f"Stock split announced: {filing.filing_date}")` See this on edgar.tools The code above detects 8-K event types by checking item numbers manually. **edgar.tools** classifies material events automatically using LLMs — earnings, acquisitions, executive changes, and more — across every 8-K as it's filed. * **[Watch 8-K filings arrive in real time with event classification →](https://app.edgar.tools/filings?utm_source=edgartools-docs&utm_medium=see-live&utm_content=eightk) ** * **[See Apple's recent material events →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=eightk) ** Includes AI-generated summaries, business descriptions, and cross-filing linkages. Free tier available. [Pricing →](https://app.edgar.tools/pricing?utm_source=edgartools-docs&utm_medium=see-live&utm_content=eightk) * * * Metadata Quick Reference ------------------------ | Property | Returns | Example | | --- | --- | --- | | `company` | Company name | `"Apple Inc."` | | `form` | Form type | `"8-K"` | | `filing_date` | Date filed with SEC | `"2024-02-01"` | | `period_of_report` | Report date | `"2024-01-31"` | | `date_of_report` | Formatted report date | `"January 31, 2024"` | | `items` | List of disclosed items | `['Item 2.02', 'Item 9.01']` | | `has_press_release` | Has EX-99 press release? | `True` | | `has_earnings` | Has parseable earnings data? | `True` | * * * Methods Quick Reference ----------------------- | Call | Returns | What it does | | --- | --- | --- | | `eight_k['2.02']` | `str` | Get item content by number | | `eight_k.press_releases` | `PressReleases` | Collection of press release exhibits | | `eight_k.earnings` | `EarningsRelease` | Parsed earnings tables from EX-99.1 | | `eight_k.income_statement` | `FinancialTable` | Income statement or None | | `eight_k.balance_sheet` | `FinancialTable` | Balance sheet or None | | `eight_k.cash_flow_statement` | `FinancialTable` | Cash flow statement or None | | `eight_k.get_income_statement()` | `DataFrame` | Income statement (empty DataFrame if missing) | | `eight_k.get_balance_sheet()` | `DataFrame` | Balance sheet (empty DataFrame if missing) | | `eight_k.get_cash_flow_statement()` | `DataFrame` | Cash flow (empty DataFrame if missing) | * * * Things to Know -------------- **Item detection is multi-tier.** EdgarTools uses document parser first (95% accuracy), falls back to text extraction for legacy SGML filings (1999-2001). **Earnings parsing requires EX-99.1.** The `has_earnings` property returns True only if Item 2.02 is present AND an EX-99.1 exhibit contains parseable tables. Some earnings 8-Ks only have narrative text. **Scale matters.** Financial tables include scale detection (thousands, millions, billions). Use `scaled_dataframe` to get values with scale applied, or check `table.scale` to apply manually. **Not all 8-Ks have XBRL.** 8-K filings typically contain only DEI (Document and Entity Information) XBRL metadata. Actual financial data is in HTML tables within EX-99 exhibits. **Press releases use pattern matching.** EdgarTools looks for EX-99, EX-99.1, EX-99.01 exhibits or exhibits with "RELEASE" in the description. Some companies use non-standard exhibit numbering. * * * Related ------- * [Stock Splits & EPS Normalization](https://edgartools.readthedocs.io/en/latest/guides/stock-splits-eps-normalization/) -- detect split announcements and normalize metrics * [Working with Filings](https://edgartools.readthedocs.io/en/latest/guides/working-with-filing/) -- general filing access patterns * [10-K Annual Reports](https://edgartools.readthedocs.io/en/latest/eightk-filings/tenk-filings.md) -- annual report parsing * [10-Q Quarterly Reports](https://edgartools.readthedocs.io/en/latest/eightk-filings/tenq-filings.md) -- quarterly report parsing Back to top --- # Common Issues & Solutions - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/resources/troubleshooting/#common-issues-solutions) Common Issues & Solutions ========================= This guide addresses the most common issues users encounter when working with edgartools and provides practical solutions. Connection and Access Issues ---------------------------- ### SEC EDGAR Access Denied **Symptom**: Receiving `403 Forbidden` errors when accessing SEC EDGAR. **Causes**: - Missing or incorrect identity information - Exceeding SEC rate limits - IP address blocked by SEC **Solutions**: 1. **Set proper identity information**: Use `set_identity` to provide your identity as required by the SEC. This requires your **name** and **email**, or just your **email**. `from edgar import set_identity set_identity("Mike McCalum mcallum@gmail.com")` 1. **Implement rate limiting**: `import time # Add delay between requests for filing in filings: # Process filing time.sleep(0.1) # 100ms delay` 2. **Use a different network** if your IP has been temporarily blocked. ### Timeout Errors **Symptom**: Requests to SEC EDGAR time out. **Solutions**: * Try again during off-peak hours (SEC EDGAR can be slow during market hours) ### SSL Certificate Errors **Symptom**: Errors like `SSL: CERTIFICATE_VERIFY_FAILED`, `certificate verify failed`, or `unable to get local issuer certificate`. **Common Causes**: - Corporate VPN with SSL inspection - Corporate proxy server - Self-signed certificates in development environments **Solutions** (in order of preference): 1. **Use OS certificate store (Recommended)**: `from edgar import configure_http # Uses your OS's trusted certificates — secure and works on corporate networks configure_http(use_system_certs=True) from edgar import Company company = Company("AAPL")` 2. **Disable SSL verification (Last resort)**: `from edgar import configure_http # WARNING: Reduces security. Only use on trusted networks. configure_http(verify_ssl=False)` 3. **Configure a proxy** if your network requires it: `from edgar import configure_http configure_http(use_system_certs=True, proxy="http://proxy.company.com:8080")` See the [SSL Configuration Guide](https://edgartools.readthedocs.io/en/latest/guides/ssl_verification/) for detailed instructions. Data Retrieval Issues --------------------- ### Filing Not Found **Symptom**: `FilingNotFoundError` when trying to access a specific filing. **Solutions**: 1. **Verify the filing exists**: `# Check if the filing exists first filings = company.get_filings(form="10-K") if filings: filing = filings.latest() else: print("No 10-K filings found")` 2. **Check for alternative form types**: `# Some companies use variant form types filings = company.get_filings(form=["10-K", "10-K/A", "10KSB"])` 3. **Expand your date range**: `filings = company.get_filings( form="10-K", start_date="2010-01-01", # Try a wider date range end_date="2023-12-31" )` ### Company Not Found **Symptom**: `CompanyNotFoundError` when trying to access a company. **Solutions**: 1. **Check ticker symbol or CIK**: `# Try using CIK instead of ticker company = Company("0000320193") # Apple Inc. CIK # Or search for the company from edgar import search_companies results = search_companies("Apple") for r in results: print(f"{r.name} - {r.ticker} - {r.cik}")` 2. **For delisted companies**, try using the CIK number directly. ### Inconsistent Financial Data Signs **Symptom**: Expense values appear negative for some companies but positive for others in cross-company analysis. **Solution**: This was resolved in edgartools 4.9.2+ through enhanced calculation weight handling. Update to the latest version: `pip install --upgrade edgartools` Major expense categories (R&D, SG&A, Marketing) are now consistently positive across companies, matching SEC CompanyFacts API behavior while preserving calculation relationships for cash flow items. ### Missing Financial Data **Symptom**: Financial statements are empty or missing expected values. **Solutions**: 1. **Check if the filing has XBRL data**: `filing = company.get_latest_filing("10-K") if filing.has_xbrl(): financials = filing.get_financials() else: print("Filing does not contain XBRL data")` 2. **Try different concept names**: `# Try alternative concept names try: revenue = income_stmt.get_value("Revenues") except: try: revenue = income_stmt.get_value("RevenueFromContractWithCustomerExcludingAssessedTax") except: revenue = income_stmt.get_value("SalesRevenueNet")` 3. **For older filings** (pre-2009), XBRL data may not be available. Parsing Issues -------------- ### HTML Parsing Errors **Symptom**: Errors when trying to extract sections from filings. **Solutions**: 1. **Access raw text instead**: `# Fall back to raw text filing_text = filing.text` 2. **Try a different filing**: `# Try the previous filing filings = company.get_filings(form="10-K") if len(filings) > 1: previous_filing = filings[1]` ### XBRL Parsing Errors **Symptom**: Errors when trying to access XBRL data. **Solutions**: 1. **Check if the filing has valid XBRL**: `if filing.has_xbrl(): try: xbrl = filing.get_xbrl() print("XBRL version:", xbrl.version) except Exception as e: print(f"XBRL parsing error: {e}")` Performance Issues ------------------ ### Slow Data Retrieval **Symptom**: Operations take a long time to complete. **Solutions**: 1. **Use local storage**: `from edgar import use_local_storage # Store filings locally use_local_storage()` 2. **Limit the number of filings**: `# Only get the 5 most recent filings filings = company.get_filings(form="10-K").head(5)` 3. **Use batch processing** for large datasets. ### Memory Issues **Symptom**: Program crashes with memory errors when processing many filings. **Solutions**: 1. **Process filings one at a time**: `for filing in filings: # Process each filing result = process_filing(filing) # Save result and free memory save_result(result) del result` 2. **Use generators instead of lists**: `def process_filings_generator(filings): for filing in filings: yield process_filing(filing) # Process one filing at a time for result in process_filings_generator(filings): save_result(result)` Installation Issues ------------------- ### Dependency Conflicts **Symptom**: Errors related to dependencies when installing or using edgartools. **Solutions**: 1. **Use a virtual environment**: `# Create a new virtual environment python -m venv edgar_env # Activate it source edgar_env/bin/activate # On Windows: edgar_env\Scripts\activate # Install edgartools pip install edgartools` 1. **Update dependencies**: `pip install --upgrade edgartools` ### Import Errors **Symptom**: `ImportError` or `ModuleNotFoundError` when importing edgartools. **Solutions**: 1. **Verify installation**: `pip show edgartools` 2. **Reinstall the package**: `pip uninstall -y edgartools pip install edgartools` 3. **Access the raw filing content**: `# Access the raw content instead html = filing.html text = filing.text` SEC Rate Limiting ----------------- ### Too Many Requests 1. **Spread requests over time**: `companies = ["AAPL", "MSFT", "GOOGL", "AMZN", "META"] results = {} for ticker in companies: company = Company(ticker) results[ticker] = company.get_latest_filing("10-K") time.sleep(1) # Wait 1 second between companies` Debugging Tips -------------- ### Enable Logging Turn on logging to get more information about what's happening: `import logging # Set up logging logging.basicConfig( level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s' ) # For even more detailed logs logging.getLogger('edgar').setLevel(logging.DEBUG)` ### Check SEC EDGAR Status The SEC EDGAR system occasionally experiences downtime or performance issues: 1. Visit the [SEC EDGAR Status page](https://www.sec.gov/edgar/filer-information/current-edgar-technical-specifications) to check for any announced issues. ### Verify Your Data Always verify the data you're working with: `# Print filing metadata to verify print(f"Filing: {filing.accession_number}") print(f"Form Type: {filing.form_type}") print(f"Filing Date: {filing.filing_date}") print(f"Has XBRL: {filing.has_xbrl()}") # Check financial statement structure financials = filing.get_financials() print(f"Available statements: {financials.available_statements()}") print(f"Available periods: {financials.get_periods()}")` Getting Help ------------ If you're still experiencing issues: 1. **Check the documentation**: Make sure you're using the API correctly. 2. **Search GitHub Issues**: Your issue may have been reported and solved already. 3. **Ask the community**: Post your question on Stack Overflow with the `edgartools` tag. 4. **Report a bug**: If you believe you've found a bug, report it on the GitHub repository with a minimal reproducible example. Common Error Messages and Their Meanings ---------------------------------------- | Error Message | Likely Cause | Solution | | --- | --- | --- | | `CompanyNotFoundError` | Invalid ticker or CIK | Verify the ticker or try using CIK | | `FilingNotFoundError` | Filing doesn't exist or is not accessible | Check form type and date range | | `XBRLNotFoundError` | Filing doesn't contain XBRL data | Try a different filing or use text extraction | | `ParsingError` | Issue parsing the filing content | Try accessing raw content instead | | `HTTPError 403` | SEC has blocked your requests | Set proper identity and respect rate limits | | `HTTPError 429` | Too many requests in a short time | Implement rate limiting and backoff | | `ConnectionError` | Network issues | Check your internet connection | | `SSLVerificationError` | Corporate VPN/proxy with SSL inspection | Use `configure_http(use_system_certs=True)` | | `CERTIFICATE_VERIFY_FAILED` | SSL certificate issues | Use `configure_http(use_system_certs=True)` | | `UnsupportedFilingTypeError` | Data Object not available for this filing type | Use generic access methods | Remember that SEC filings can vary significantly in structure and content, especially across different years and companies. Always implement robust error handling in your code to deal with these variations. Back to top --- # Common Issues & Solutions - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/resources/troubleshooting/#common-issues-solutions) Common Issues & Solutions ========================= This guide addresses the most common issues users encounter when working with edgartools and provides practical solutions. Connection and Access Issues ---------------------------- ### SEC EDGAR Access Denied **Symptom**: Receiving `403 Forbidden` errors when accessing SEC EDGAR. **Causes**: - Missing or incorrect identity information - Exceeding SEC rate limits - IP address blocked by SEC **Solutions**: 1. **Set proper identity information**: Use `set_identity` to provide your identity as required by the SEC. This requires your **name** and **email**, or just your **email**. `from edgar import set_identity set_identity("Mike McCalum mcallum@gmail.com")` 1. **Implement rate limiting**: `import time # Add delay between requests for filing in filings: # Process filing time.sleep(0.1) # 100ms delay` 2. **Use a different network** if your IP has been temporarily blocked. ### Timeout Errors **Symptom**: Requests to SEC EDGAR time out. **Solutions**: * Try again during off-peak hours (SEC EDGAR can be slow during market hours) ### SSL Certificate Errors **Symptom**: Errors like `SSL: CERTIFICATE_VERIFY_FAILED`, `certificate verify failed`, or `unable to get local issuer certificate`. **Common Causes**: - Corporate VPN with SSL inspection - Corporate proxy server - Self-signed certificates in development environments **Solutions** (in order of preference): 1. **Use OS certificate store (Recommended)**: `from edgar import configure_http # Uses your OS's trusted certificates — secure and works on corporate networks configure_http(use_system_certs=True) from edgar import Company company = Company("AAPL")` 2. **Disable SSL verification (Last resort)**: `from edgar import configure_http # WARNING: Reduces security. Only use on trusted networks. configure_http(verify_ssl=False)` 3. **Configure a proxy** if your network requires it: `from edgar import configure_http configure_http(use_system_certs=True, proxy="http://proxy.company.com:8080")` See the [SSL Configuration Guide](https://edgartools.readthedocs.io/en/stable/guides/ssl_verification/) for detailed instructions. Data Retrieval Issues --------------------- ### Filing Not Found **Symptom**: `FilingNotFoundError` when trying to access a specific filing. **Solutions**: 1. **Verify the filing exists**: `# Check if the filing exists first filings = company.get_filings(form="10-K") if filings: filing = filings.latest() else: print("No 10-K filings found")` 2. **Check for alternative form types**: `# Some companies use variant form types filings = company.get_filings(form=["10-K", "10-K/A", "10KSB"])` 3. **Expand your date range**: `filings = company.get_filings( form="10-K", start_date="2010-01-01", # Try a wider date range end_date="2023-12-31" )` ### Company Not Found **Symptom**: `CompanyNotFoundError` when trying to access a company. **Solutions**: 1. **Check ticker symbol or CIK**: `# Try using CIK instead of ticker company = Company("0000320193") # Apple Inc. CIK # Or search for the company from edgar import search_companies results = search_companies("Apple") for r in results: print(f"{r.name} - {r.ticker} - {r.cik}")` 2. **For delisted companies**, try using the CIK number directly. ### Inconsistent Financial Data Signs **Symptom**: Expense values appear negative for some companies but positive for others in cross-company analysis. **Solution**: This was resolved in edgartools 4.9.2+ through enhanced calculation weight handling. Update to the latest version: `pip install --upgrade edgartools` Major expense categories (R&D, SG&A, Marketing) are now consistently positive across companies, matching SEC CompanyFacts API behavior while preserving calculation relationships for cash flow items. ### Missing Financial Data **Symptom**: Financial statements are empty or missing expected values. **Solutions**: 1. **Check if the filing has XBRL data**: `filing = company.get_latest_filing("10-K") if filing.has_xbrl(): financials = filing.get_financials() else: print("Filing does not contain XBRL data")` 2. **Try different concept names**: `# Try alternative concept names try: revenue = income_stmt.get_value("Revenues") except: try: revenue = income_stmt.get_value("RevenueFromContractWithCustomerExcludingAssessedTax") except: revenue = income_stmt.get_value("SalesRevenueNet")` 3. **For older filings** (pre-2009), XBRL data may not be available. Parsing Issues -------------- ### HTML Parsing Errors **Symptom**: Errors when trying to extract sections from filings. **Solutions**: 1. **Access raw text instead**: `# Fall back to raw text filing_text = filing.text` 2. **Try a different filing**: `# Try the previous filing filings = company.get_filings(form="10-K") if len(filings) > 1: previous_filing = filings[1]` ### XBRL Parsing Errors **Symptom**: Errors when trying to access XBRL data. **Solutions**: 1. **Check if the filing has valid XBRL**: `if filing.has_xbrl(): try: xbrl = filing.get_xbrl() print("XBRL version:", xbrl.version) except Exception as e: print(f"XBRL parsing error: {e}")` Performance Issues ------------------ ### Slow Data Retrieval **Symptom**: Operations take a long time to complete. **Solutions**: 1. **Use local storage**: `from edgar import use_local_storage # Store filings locally use_local_storage()` 2. **Limit the number of filings**: `# Only get the 5 most recent filings filings = company.get_filings(form="10-K").head(5)` 3. **Use batch processing** for large datasets. ### Memory Issues **Symptom**: Program crashes with memory errors when processing many filings. **Solutions**: 1. **Process filings one at a time**: `for filing in filings: # Process each filing result = process_filing(filing) # Save result and free memory save_result(result) del result` 2. **Use generators instead of lists**: `def process_filings_generator(filings): for filing in filings: yield process_filing(filing) # Process one filing at a time for result in process_filings_generator(filings): save_result(result)` Installation Issues ------------------- ### Dependency Conflicts **Symptom**: Errors related to dependencies when installing or using edgartools. **Solutions**: 1. **Use a virtual environment**: `# Create a new virtual environment python -m venv edgar_env # Activate it source edgar_env/bin/activate # On Windows: edgar_env\Scripts\activate # Install edgartools pip install edgartools` 1. **Update dependencies**: `pip install --upgrade edgartools` ### Import Errors **Symptom**: `ImportError` or `ModuleNotFoundError` when importing edgartools. **Solutions**: 1. **Verify installation**: `pip show edgartools` 2. **Reinstall the package**: `pip uninstall -y edgartools pip install edgartools` 3. **Access the raw filing content**: `# Access the raw content instead html = filing.html text = filing.text` SEC Rate Limiting ----------------- ### Too Many Requests 1. **Spread requests over time**: `companies = ["AAPL", "MSFT", "GOOGL", "AMZN", "META"] results = {} for ticker in companies: company = Company(ticker) results[ticker] = company.get_latest_filing("10-K") time.sleep(1) # Wait 1 second between companies` Debugging Tips -------------- ### Enable Logging Turn on logging to get more information about what's happening: `import logging # Set up logging logging.basicConfig( level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s' ) # For even more detailed logs logging.getLogger('edgar').setLevel(logging.DEBUG)` ### Check SEC EDGAR Status The SEC EDGAR system occasionally experiences downtime or performance issues: 1. Visit the [SEC EDGAR Status page](https://www.sec.gov/edgar/filer-information/current-edgar-technical-specifications) to check for any announced issues. ### Verify Your Data Always verify the data you're working with: `# Print filing metadata to verify print(f"Filing: {filing.accession_number}") print(f"Form Type: {filing.form_type}") print(f"Filing Date: {filing.filing_date}") print(f"Has XBRL: {filing.has_xbrl()}") # Check financial statement structure financials = filing.get_financials() print(f"Available statements: {financials.available_statements()}") print(f"Available periods: {financials.get_periods()}")` Getting Help ------------ If you're still experiencing issues: 1. **Check the documentation**: Make sure you're using the API correctly. 2. **Search GitHub Issues**: Your issue may have been reported and solved already. 3. **Ask the community**: Post your question on Stack Overflow with the `edgartools` tag. 4. **Report a bug**: If you believe you've found a bug, report it on the GitHub repository with a minimal reproducible example. Common Error Messages and Their Meanings ---------------------------------------- | Error Message | Likely Cause | Solution | | --- | --- | --- | | `CompanyNotFoundError` | Invalid ticker or CIK | Verify the ticker or try using CIK | | `FilingNotFoundError` | Filing doesn't exist or is not accessible | Check form type and date range | | `XBRLNotFoundError` | Filing doesn't contain XBRL data | Try a different filing or use text extraction | | `ParsingError` | Issue parsing the filing content | Try accessing raw content instead | | `HTTPError 403` | SEC has blocked your requests | Set proper identity and respect rate limits | | `HTTPError 429` | Too many requests in a short time | Implement rate limiting and backoff | | `ConnectionError` | Network issues | Check your internet connection | | `SSLVerificationError` | Corporate VPN/proxy with SSL inspection | Use `configure_http(use_system_certs=True)` | | `CERTIFICATE_VERIFY_FAILED` | SSL certificate issues | Use `configure_http(use_system_certs=True)` | | `UnsupportedFilingTypeError` | Data Object not available for this filing type | Use generic access methods | Remember that SEC filings can vary significantly in structure and content, especially across different years and companies. Always implement robust error handling in your code to deal with these variations. Back to top --- # Extract Financial Statements - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/guides/extract-statements/#extract-financial-statements-from-sec-filings-with-python) Extract Financial Statements from SEC Filings with Python ========================================================= Learn how to extract and work with financial statements from SEC filings using EdgarTools' powerful XBRL processing capabilities. Prerequisites ------------- * Basic understanding of financial statements (balance sheet, income statement, cash flow) * Familiarity with [finding companies](https://edgartools.readthedocs.io/en/stable/guides/finding-companies/) and [searching filings](https://edgartools.readthedocs.io/en/stable/guides/searching-filings/) Quick Start: Single Period Statements ------------------------------------- ### Get Latest Financial Statements The fastest way to get financial statements is using the Company.financials property: `from edgar import Company # Get Apple's latest financials company = Company("AAPL") financials = company.get_financials() # Access individual statements balance_sheet = financials.balance_sheet income_statement = financials.income_statement() cash_flow = financials.cashflow_statement()` ### Alternative: From Specific Filing For more control, extract statements from a specific filing: `from edgar import Company # Get a specific filing company = Company("AAPL") filing = company.get_filings(form="10-K").latest() # Parse XBRL data xbrl = filing.xbrl() # Access statements through the user-friendly API statements = xbrl.statements # Display financial statements balance_sheet = statements.balance_sheet() income_statement = statements.income_statement() cash_flow = statements.cashflow_statement() print(balance_sheet) # Rich formatted output` ### Enhanced Dimensional Display EdgarTools now automatically surfaces rich dimensional segment data in financial statements when available: `# Get Microsoft's income statement - now shows product/service breakdowns company = Company("MSFT") xbrl = company.get_filings(form="10-K").latest().xbrl() income_stmt = xbrl.statements.income_statement() print(income_stmt) # Output shows both summary revenue AND detailed breakdowns: # - Product revenue: $63.9B # - Service revenue: $217.8B # - Business segment details (LinkedIn: $17.8B, Gaming: $23.5B, etc.)` **Control Dimensional Display:** `# Default behavior - includes dimensional segment data df_enhanced = income_stmt.to_dataframe() # 48 rows for Microsoft print(f"Enhanced view: {len(df_enhanced)} rows") # Standard view - face presentation only df_standard = income_stmt.to_dataframe(view="standard") # 21 rows print(f"Standard view: {len(df_standard)} rows") # Summary view - non-dimensional totals only df_summary = income_stmt.to_dataframe(view="summary") print(f"Summary view: {len(df_summary)} rows")` **What Gets Enhanced:** - Product/service revenue breakdowns - Geographic segment data \- Business unit financial details - Any ProductOrServiceAxis dimensional facts This enhancement works automatically across companies that provide segment data in their XBRL filings, including Microsoft, Apple, Amazon, Google, and many others. Understanding Statement Hierarchy --------------------------------- Every financial statement has a tree structure — revenue breaks down into product vs service revenue, operating expenses break down into R&D, SG&A, etc. When you call `to_dataframe()`, EdgarTools includes hierarchy columns that let you navigate these relationships programmatically. ### Hierarchy Columns The DataFrame returned by `to_dataframe()` includes these columns for understanding structure: | Column | Description | Example | | --- | --- | --- | | `level` | Nesting depth (0=root, 1=section, 2=line item, 3=sub-item) | `2` | | `abstract` | True for section headers, False for data rows | `True` | | `parent_concept` | Calculation tree parent — the metric this rolls up to | `us-gaap_Revenue` | | `parent_abstract_concept` | Presentation tree parent — the section header above | `us-gaap_RevenueAbstract` | ### Example: Apple Revenue Breakdown `from edgar import Company company = Company("AAPL") filing = company.get_filings(form="10-K").latest() xbrl = filing.xbrl() income = xbrl.statements.income_statement() df = income.to_dataframe() # Show hierarchy for the first 10 rows print(df[['label', 'level', 'parent_concept', 'parent_abstract_concept']].head(10))` Output shows the tree structure: `label level parent_concept parent_abstract_concept 0 Net sales: 1 None us-gaap_RevenueFromCont... 1 Products 2 us-gaap_Revenue... us-gaap_RevenueFromCont... 2 Services 2 us-gaap_Revenue... us-gaap_RevenueFromCont... 3 Total net sales 1 None us-gaap_RevenueFromCont... 4 Cost of sales: 1 None us-gaap_CostOfGoodsSold... 5 Products 2 us-gaap_CostOf... us-gaap_CostOfGoodsSold... 6 Services 2 us-gaap_CostOf... us-gaap_CostOfGoodsSold...` ### Building a Parent-Child Tree Use the `level` column to reconstruct the hierarchy: `# Get all non-abstract rows with their hierarchy info data_rows = df[~df['abstract']] hierarchy = data_rows[['label', 'level', 'parent_concept', 'parent_abstract_concept']].copy() # Show indented tree for _, row in hierarchy.iterrows(): indent = ' ' * row['level'] print(f"{indent}{row['label']}")` ### Extracting Revenue Segments To get just the revenue breakdown with parent-child relationships: `# Find revenue-related rows using parent_concept revenue_concept = 'us-gaap_RevenueFromContractWithCustomerExcludingAssessedTax' # All rows whose parent is Revenue (these are the segments) segments = df[df['parent_concept'] == revenue_concept] print(segments[['label', 'level']].to_string(index=False))` ### When to Use `query()` vs `Statement` | Goal | Use | Why | | --- | --- | --- | | Hierarchical data with parent-child | `statement.to_dataframe()` | Preserves presentation tree structure | | Flat fact extraction with filters | `xbrl.query()` | Efficient for specific concept lookups | | Revenue segment tree | `statement.to_dataframe()` | Has `level`, `parent_concept` columns | | Specific dimensional breakdowns | `statement.to_dataframe(view="detailed")` | Includes all dimensional data | | Cross-filing fact comparison | `xbrl.query()` | Works across periods and filings | ### Controlling Detail Level with `view` The `view` parameter controls how much dimensional data appears: `# Standard: face presentation only (what you see in SEC Viewer) df_standard = income.to_dataframe(view="standard") # Detailed: includes all dimensional breakdowns (segments, geography, etc.) df_detailed = income.to_dataframe(view="detailed") # Summary: non-dimensional totals only df_summary = income.to_dataframe(view="summary")` See the [Dimension Handling Guide](https://edgartools.readthedocs.io/en/stable/xbrl/concepts/dimension-handling/) for more on controlling dimensional data. * * * Standardized Financial Data Access ---------------------------------- EdgarTools automatically standardizes XBRL data across companies, mapping ~2,000 different XBRL tags to 95 consistent concepts. This means you can compare Apple's revenue with Tesla's revenue using the same API, even though they use different underlying XBRL concepts. **Why this matters:** - Companies use different XBRL tags for the same concept (e.g., "Revenues", "RevenueFromContractWithCustomer", "SalesRevenueNet") - EdgarTools normalizes these to standard concepts like "Revenue" - Cross-company analysis becomes trivial For the complete list of 95 standard concepts and their mappings, see [Standardization Concepts Reference](https://edgartools.readthedocs.io/en/stable/xbrl/concepts/standardization/) . ### Simple Metric Extraction The easiest way to get key financial metrics is using the standardized accessor methods: `from edgar import Company # Get a company's financials company = Company("AAPL") financials = company.get_financials() # Extract key metrics directly - these work across all companies! revenue = financials.get_revenue() net_income = financials.get_net_income() total_assets = financials.get_total_assets() print(f"Revenue: ${revenue:,.0f}") print(f"Net Income: ${net_income:,.0f}") print(f"Total Assets: ${total_assets:,.0f}")` This simple API works consistently across all companies, regardless of their custom XBRL concepts! ### Available Standardized Methods All methods support historical data via the `period_offset` parameter: `# Income Statement Metrics revenue_current = financials.get_revenue() # Current period revenue_previous = financials.get_revenue(1) # Previous period net_income = financials.get_net_income() # Balance Sheet Metrics total_assets = financials.get_total_assets() total_liabilities = financials.get_total_liabilities() stockholders_equity = financials.get_stockholders_equity() current_assets = financials.get_current_assets() current_liabilities = financials.get_current_liabilities() # Cash Flow Metrics operating_cash_flow = financials.get_operating_cash_flow() capital_expenditures = financials.get_capital_expenditures() free_cash_flow = financials.get_free_cash_flow() # Calculated automatically` ### Comprehensive Financial Analysis Get all key metrics at once with automatic ratio calculations: `# Get comprehensive metrics dictionary metrics = financials.get_financial_metrics() # All the standard metrics are available print(f"Revenue: ${metrics['revenue']:,.0f}") print(f"Net Income: ${metrics['net_income']:,.0f}") print(f"Total Assets: ${metrics['total_assets']:,.0f}") # Plus calculated ratios print(f"Current Ratio: {metrics['current_ratio']:.2f}") print(f"Debt to Assets: {metrics['debt_to_assets']:.2f}") print(f"Free Cash Flow: ${metrics['free_cash_flow']:,.0f}")` ### Cross-Company Analysis Made Simple Now comparing multiple companies is trivial: `companies = ["AAPL", "MSFT", "GOOGL", "AMZN", "META"] print("Company\t\tRevenue\t\tNet Income\tTotal Assets") print("-" * 60) for ticker in companies: company = Company(ticker) financials = company.get_financials() if financials: revenue = financials.get_revenue() net_income = financials.get_net_income() total_assets = financials.get_total_assets() print(f"{ticker}\t\t${revenue/1e9:.1f}B\t\t${net_income/1e9:.1f}B\t\t${total_assets/1e9:.1f}B")` ### Tesla Custom Concepts - No Problem! The standardized methods automatically handle companies with custom concepts like Tesla: `# Works even with companies that use non-standard XBRL concepts tesla = Company("TSLA") tsla_financials = tesla.get_financials() # These work despite Tesla's custom concepts tsla_revenue = tsla_financials.get_revenue() tsla_net_income = tsla_financials.get_net_income() print(f"Tesla Revenue: ${tsla_revenue:,.0f}") print(f"Tesla Net Income: ${tsla_net_income:,.0f}")` ### Growth Analysis with Historical Data Calculate growth rates using the `period_offset` parameter: `# Get current and previous year data current_revenue = financials.get_revenue(0) # Current period previous_revenue = financials.get_revenue(1) # Previous period if current_revenue and previous_revenue: growth_rate = (current_revenue - previous_revenue) / previous_revenue * 100 print(f"Revenue Growth: {growth_rate:.1f}%") # Same pattern works for any metric current_ni = financials.get_net_income(0) previous_ni = financials.get_net_income(1) if current_ni and previous_ni: ni_growth = (current_ni - previous_ni) / previous_ni * 100 print(f"Net Income Growth: {ni_growth:.1f}%")` Multi-Period Analysis --------------------- ### Method 1: Using MultiFinancials Get financials across multiple years for trend analysis: `from edgar import Company, MultiFinancials # Get multiple years of 10-K filings company = Company("AAPL") filings = company.get_filings(form="10-K").head(3) # Last 3 annual reports # Create multi-period financials multi_financials = MultiFinancials.extract(filings) # Access statements spanning multiple years balance_sheet = multi_financials.balance_sheet() income_statement = multi_financials.income_statement() cash_flow = multi_financials.cashflow_statement() # Use view="detailed" to include dimensional breakdowns (e.g., cost by segment) income_detailed = multi_financials.income_statement(view="detailed") print("Multi-Year Income Statement:") print(income_statement)` ### Method 2: Using XBRL Stitching For more advanced multi-period analysis with intelligent period matching: `from edgar import Company from edgar.xbrl import XBRLS # Get multiple filings for trend analysis company = Company("AAPL") filings = company.get_filings(form="10-K").head(3) # Create stitched view across multiple filings xbrls = XBRLS.from_filings(filings) # Access stitched statements stitched_statements = xbrls.statements # Display multi-period statements with intelligent period selection income_trend = stitched_statements.income_statement() balance_sheet_trend = stitched_statements.balance_sheet() cashflow_trend = stitched_statements.cashflow_statement() print("Three-Year Revenue Trend:") revenue_trend = income_trend.to_dataframe() revenue_row = revenue_trend.loc[revenue_trend['label'] == 'Revenue'] print(revenue_row)` **Dimensional Data in Stitching:** By default, stitching uses traditional statement structures for performance and compatibility. Use the `view` parameter to control dimensional data: `# Default stitching - standard face presentation for multi-period consistency income_stmt = stitched_statements.income_statement() # Clean, focused view # Include dimensional breakdowns (e.g., cost of operations by segment) income_stmt_detailed = stitched_statements.income_statement(view="detailed") # Summary view - non-dimensional totals only income_stmt_summary = stitched_statements.income_statement(view="summary")` **When to Use Each View:** * **`"standard"`** (default): Best for trend analysis, ratios, and cross-period comparisons * **`"detailed"`**: Use when you need segment data across periods (e.g., cost breakdowns by product line) * **`"summary"`**: Quick overview of main line items only Working with Individual Statements ---------------------------------- ### Balance Sheet Analysis `# Get balance sheet balance_sheet = statements.balance_sheet() # Convert to DataFrame for analysis bs_df = balance_sheet.to_dataframe() # Extract key balance sheet items total_assets = bs_df[bs_df['label'] == 'Total Assets'] total_liabilities = bs_df[bs_df['label'] == 'Total Liabilities'] shareholders_equity = bs_df[bs_df['label'] == "Total Stockholders' Equity"] print("Balance Sheet Summary:") print(f"Total Assets: ${total_assets.iloc[0, -1]/1e9:.1f}B") print(f"Total Liabilities: ${total_liabilities.iloc[0, -1]/1e9:.1f}B") print(f"Shareholders' Equity: ${shareholders_equity.iloc[0, -1]/1e9:.1f}B") # Calculate debt-to-equity ratio debt_to_equity = total_liabilities.iloc[0, -1] / shareholders_equity.iloc[0, -1] print(f"Debt-to-Equity Ratio: {debt_to_equity:.2f}")` ### Income Statement Analysis `# Get income statement income_statement = statements.income_statement() # Convert to DataFrame is_df = income_statement.to_dataframe() # Extract key income statement items revenue = is_df[is_df['label'] == 'Revenue'] gross_profit = is_df[is_df['label'] == 'Gross Profit'] operating_income = is_df[is_df['label'] == 'Operating Income'] net_income = is_df[is_df['label'] == 'Net Income'] print("Income Statement Analysis:") print(f"Revenue: ${revenue.iloc[0, -1]/1e9:.1f}B") print(f"Gross Profit: ${gross_profit.iloc[0, -1]/1e9:.1f}B") print(f"Operating Income: ${operating_income.iloc[0, -1]/1e9:.1f}B") print(f"Net Income: ${net_income.iloc[0, -1]/1e9:.1f}B") # Calculate margins gross_margin = (gross_profit.iloc[0, -1] / revenue.iloc[0, -1]) * 100 operating_margin = (operating_income.iloc[0, -1] / revenue.iloc[0, -1]) * 100 net_margin = (net_income.iloc[0, -1] / revenue.iloc[0, -1]) * 100 print(f"\nMargin Analysis:") print(f"Gross Margin: {gross_margin:.1f}%") print(f"Operating Margin: {operating_margin:.1f}%") print(f"Net Margin: {net_margin:.1f}%")` ### Cash Flow Analysis `# Get cash flow statement cash_flow = statements.cashflow_statement() # Convert to DataFrame cf_df = cash_flow.to_dataframe() # Extract cash flow components operating_cf = cf_df[cf_df['label'] == 'Net Cash from Operating Activities'] investing_cf = cf_df[cf_df['label'] == 'Net Cash from Investing Activities'] financing_cf = cf_df[cf_df['label'] == 'Net Cash from Financing Activities'] print("Cash Flow Analysis:") print(f"Operating Cash Flow: ${operating_cf.iloc[0, -1]/1e9:.1f}B") print(f"Investing Cash Flow: ${investing_cf.iloc[0, -1]/1e9:.1f}B") print(f"Financing Cash Flow: ${financing_cf.iloc[0, -1]/1e9:.1f}B") # Calculate free cash flow (Operating CF - Capital Expenditures) capex = cf_df[cf_df['label'].str.contains('Capital Expenditures', case=False, na=False)] if not capex.empty: free_cash_flow = operating_cf.iloc[0, -1] + capex.iloc[0, -1] # CapEx is usually negative print(f"Free Cash Flow: ${free_cash_flow/1e9:.1f}B")` Advanced Statement Customization -------------------------------- ### Period Views and Formatting `# Get available period views for income statement period_views = statements.get_period_views("IncomeStatement") print("Available period views:") for view in period_views: print(f"- {view['name']}: {view['description']}") # Render with specific period view annual_comparison = statements.income_statement(period_view="Annual Comparison") quarterly_comparison = statements.income_statement(period_view="Quarterly Comparison") # Show full date ranges for duration periods income_with_dates = statements.income_statement(show_date_range=True) print("Income Statement with Date Ranges:") print(income_with_dates)` ### Standardized vs Company-Specific Labels When using stitched statements (multi-period analysis), you can control label standardization: `from edgar import Company from edgar.xbrl import XBRLS # Get multiple filings for stitched analysis company = Company("AAPL") filings = company.get_filings(form="10-K").head(3) xbrls = XBRLS.from_filings(filings) stitched = xbrls.statements # Get income statement with standard_concept metadata (default) income = stitched.income_statement(standard=True) # Labels always show original company presentation # Use standard_concept for cross-company analysis df = income.to_dataframe() print("Labels with Standard Concept Mapping:") print(df[['label', 'standard_concept']].head(10)) # Aggregate by standard concept for comparison standardized = df.groupby('standard_concept')[df.columns[2:4]].sum() print("\nAggregated by Standard Concept:") print(standardized.head(10))` > **Note**: The `standard=True` parameter adds `standard_concept` metadata for cross-company analysis. Labels always preserve the company's original presentation. Cross-Company Analysis ---------------------- ### Compare Multiple Companies (Updated with New API!) `import pandas as pd def get_key_metrics(ticker): """Extract key financial metrics for a company using new standardized methods.""" try: company = Company(ticker) financials = company.get_financials() if not financials: return None # Use the new standardized accessor methods - much simpler! return { 'ticker': ticker, 'revenue': financials.get_revenue(), 'net_income': financials.get_net_income(), 'total_assets': financials.get_total_assets(), 'operating_cf': financials.get_operating_cash_flow(), 'free_cf': financials.get_free_cash_flow() } except Exception as e: print(f"Error processing {ticker}: {e}") return None # Analyze multiple companies tech_companies = ['AAPL', 'MSFT', 'GOOGL', 'AMZN', 'META'] metrics = [] for ticker in tech_companies: result = get_key_metrics(ticker) if result: metrics.append(result) # Create comparison DataFrame comparison_df = pd.DataFrame(metrics) # Convert to billions and calculate ratios comparison_df['revenue_b'] = comparison_df['revenue'] / 1e9 comparison_df['net_income_b'] = comparison_df['net_income'] / 1e9 comparison_df['net_margin'] = (comparison_df['net_income'] / comparison_df['revenue']) * 100 comparison_df['roa'] = (comparison_df['net_income'] / comparison_df['total_assets']) * 100 print("Tech Giants Comparison:") print(comparison_df[['ticker', 'revenue_b', 'net_income_b', 'net_margin', 'roa']].round(1))` The new standardized methods make cross-company analysis much more reliable and easier to implement! Notes and Disclosures --------------------- XBRL filings contain notes and disclosure sections beyond the primary financial statements. Access them with convenience methods: `xbrl = filing.xbrl() # Browse all note sections (e.g., accounting policies, segment data) for note in xbrl.notes(): print(note.title) # Browse all disclosure sections (e.g., revenue disaggregation, debt details) for disc in xbrl.disclosures(): print(disc.title)` These return the same statement objects as `xbrl.statements`, filtered to notes and disclosures respectively. Advanced XBRL Features ---------------------- ### Access Raw XBRL Facts `# Access the facts API for detailed XBRL data facts = xbrl.facts # Query facts by concept revenue_facts = facts.query().by_concept('Revenue').to_dataframe() print("Revenue facts across all periods:") print(revenue_facts[['concept', 'label', 'period', 'value']]) # Search for specific concepts earnings_facts = facts.search_facts("Earnings Per Share") print("EPS-related facts:") print(earnings_facts[['concept', 'label', 'value']]) # Get facts by statement type balance_sheet_facts = facts.query().by_statement_type('BalanceSheet').to_dataframe() print(f"Found {len(balance_sheet_facts)} balance sheet facts")` ### Time Series Analysis `# Get time series data for specific concepts revenue_series = facts.time_series('Revenue') net_income_series = facts.time_series('Net Income') print("Revenue Time Series:") print(revenue_series) # Convert to DataFrame for analysis import pandas as pd ts_df = pd.DataFrame({ 'revenue': revenue_series, 'net_income': net_income_series }) # Calculate growth rates ts_df['revenue_growth'] = ts_df['revenue'].pct_change() * 100 ts_df['income_growth'] = ts_df['net_income'].pct_change() * 100 print("Growth Analysis:") print(ts_df[['revenue_growth', 'income_growth']].round(1))` ### Dimensional Analysis `# Query facts by dimensions (if available) segment_facts = facts.query().by_dimension('Segment').to_dataframe() if not segment_facts.empty: print("Segment-specific financial data:") print(segment_facts[['concept', 'label', 'dimension_value', 'value']].head()) # Get facts by geographic dimension geographic_facts = facts.query().by_dimension('Geography').to_dataframe() if not geographic_facts.empty: print("Geographic breakdown:") print(geographic_facts[['concept', 'dimension_value', 'value']].head())` Export and Integration ---------------------- ### Export to Different Formats `# Export statements to various formats income_statement = statements.income_statement() # Export to pandas DataFrame df = income_statement.to_dataframe() # Export to markdown markdown_text = income_statement.render().to_markdown() # Save to CSV df.to_csv('apple_income_statement.csv', index=False) # Save markdown to file with open('apple_income_statement.md', 'w') as f: f.write(markdown_text) print("Statements exported to CSV and Markdown")` ### Integration with Analysis Libraries `import matplotlib.pyplot as plt import seaborn as sns # Get multi-period data filings = company.get_filings(form="10-K").head(5) multi_financials = MultiFinancials.extract(filings) income_df = multi_financials.income.to_dataframe() # Extract revenue data for plotting revenue_data = income_df[income_df['label'] == 'Revenue'].iloc[0, 1:].astype(float) periods = revenue_data.index # Create visualization plt.figure(figsize=(10, 6)) plt.plot(periods, revenue_data / 1e9, marker='o', linewidth=2) plt.title('Apple Revenue Trend (5 Years)') plt.xlabel('Period') plt.ylabel('Revenue (Billions USD)') plt.xticks(rotation=45) plt.grid(True, alpha=0.3) plt.tight_layout() plt.show() # Calculate year-over-year growth revenue_growth = revenue_data.pct_change() * 100 print("Year-over-Year Revenue Growth:") for period, growth in revenue_growth.dropna().items(): print(f"{period}: {growth:.1f}%")` Performance Optimization ------------------------ ### Efficient Multi-Company Analysis `# Efficient batch processing def batch_analyze_companies(tickers, max_workers=5): """Analyze multiple companies efficiently.""" from concurrent.futures import ThreadPoolExecutor def analyze_single(ticker): try: company = Company(ticker) financials = company.financials return { 'ticker': ticker, 'revenue': financials.income.loc['Revenue'].iloc[0], 'assets': financials.balance_sheet.loc['Total Assets'].iloc[0] } except Exception as e: return {'ticker': ticker, 'error': str(e)} with ThreadPoolExecutor(max_workers=max_workers) as executor: results = list(executor.map(analyze_single, tickers)) return [r for r in results if 'error' not in r] # Analyze S&P 100 companies efficiently sp100_sample = ['AAPL', 'MSFT', 'GOOGL', 'AMZN', 'META', 'TSLA', 'NVDA', 'JPM'] results = batch_analyze_companies(sp100_sample) comparison_df = pd.DataFrame(results) print("Batch Analysis Results:") print(comparison_df.head())` ### Caching for Repeated Analysis `# Cache XBRL data for repeated use company = Company("AAPL") filing = company.get_filings(form="10-K").latest() # Parse once, use multiple times xbrl = filing.xbrl() # Perform different analyses on same data balance_sheet = xbrl.statements.balance_sheet() income_statement = xbrl.statements.income_statement() cash_flow = xbrl.statements.cashflow_statement() # Access facts for custom queries facts = xbrl.facts revenue_facts = facts.query().by_concept('Revenue').to_dataframe() margin_facts = facts.search_facts("margin")` Common Patterns and Best Practices ---------------------------------- ### Robust Financial Metric Extraction `def safe_extract_metric(statement_df, label, column=-1, default=None): """Safely extract a metric from financial statement DataFrame.""" try: rows = statement_df[statement_df['label'].str.contains(label, case=False, na=False)] if not rows.empty: return rows.iloc[0, column] return default except Exception: return default # Use for robust metric extraction income_df = statements.income_statement().to_dataframe() revenue = safe_extract_metric(income_df, 'Revenue') net_income = safe_extract_metric(income_df, 'Net Income') operating_income = safe_extract_metric(income_df, 'Operating Income') if revenue and net_income: net_margin = (net_income / revenue) * 100 print(f"Net Margin: {net_margin:.1f}%")` ### Handle Missing or Inconsistent Data `def get_financial_metrics(company_ticker): """Get financial metrics with error handling.""" try: company = Company(company_ticker) financials = company.financials metrics = {} # Try to get income statement metrics try: income = financials.income metrics['revenue'] = income.loc['Revenue'].iloc[0] if 'Revenue' in income.index else None metrics['net_income'] = income.loc['Net Income'].iloc[0] if 'Net Income' in income.index else None except Exception as e: print(f"Income statement error for {company_ticker}: {e}") # Try to get balance sheet metrics try: balance_sheet = financials.balance_sheet metrics['total_assets'] = balance_sheet.loc['Total Assets'].iloc[0] if 'Total Assets' in balance_sheet.index else None except Exception as e: print(f"Balance sheet error for {company_ticker}: {e}") return metrics except Exception as e: print(f"Company error for {company_ticker}: {e}") return {} # Test with various companies test_companies = ['AAPL', 'INVALID_TICKER', 'MSFT'] for ticker in test_companies: metrics = get_financial_metrics(ticker) if metrics: print(f"{ticker}: {metrics}")` Troubleshooting Common Issues ----------------------------- ### Statement Not Available `# Check what statements are available try: statements = xbrl.statements available_statements = statements.available_statements() print(f"Available statements: {available_statements}") # Try alternative statement access if 'IncomeStatement' in available_statements: income = statements.income_statement() elif 'ComprehensiveIncome' in available_statements: income = statements['ComprehensiveIncome'] else: print("No income statement available") except Exception as e: print(f"Error accessing statements: {e}")` ### Period Selection Issues `# Check available periods reporting_periods = xbrl.reporting_periods print("Available reporting periods:") for period in reporting_periods[:5]: # Show first 5 print(f"- {period['date']} ({period['type']}): {period.get('duration', 'N/A')} days") # Handle quarterly vs annual periods if any(p.get('duration', 0) < 120 for p in reporting_periods): print("Quarterly periods detected") quarterly_income = statements.income_statement(period_view="Quarterly Comparison") else: print("Annual periods only") annual_income = statements.income_statement(period_view="Annual Comparison")` Next Steps ---------- Now that you can extract financial statements, explore these advanced topics: * **[XBRL Documentation Hub](https://edgartools.readthedocs.io/en/stable/xbrl/) ** - Central navigation for all XBRL documentation * **[Multi-Period Analysis](https://edgartools.readthedocs.io/en/stable/xbrl/guides/multi-period-analysis/) ** - Compare financials across multiple years * **[Choosing the Right API](https://edgartools.readthedocs.io/en/stable/xbrl/getting-started/choosing-the-right-api/) ** - Decision guide for which API to use * **[Dimension Handling Guide](https://edgartools.readthedocs.io/en/stable/xbrl/concepts/dimension-handling/) ** - Understanding dimensional data (segments, breakdowns) * **[Standardization Concepts](https://edgartools.readthedocs.io/en/stable/xbrl/concepts/standardization/) ** - 95 standard concepts for cross-company comparison Related Documentation --------------------- * **[Getting XBRL from Filings](https://edgartools.readthedocs.io/en/stable/getting-xbrl/) ** - Original XBRL documentation * **[Company Financials](https://edgartools.readthedocs.io/en/stable/guides/company-financials.md) ** - Company financials API * **[XBRL API Reference](https://edgartools.readthedocs.io/en/stable/api/xbrl/) ** - Complete XBRL class documentation * **[StatementType Quick Reference](https://edgartools.readthedocs.io/en/stable/StatementType-Quick-Reference/) ** - Statement type enums and API comparison Back to top --- # Extract Financial Statements - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/guides/extract-statements/#extract-financial-statements-from-sec-filings-with-python) Extract Financial Statements from SEC Filings with Python ========================================================= Learn how to extract and work with financial statements from SEC filings using EdgarTools' powerful XBRL processing capabilities. Prerequisites ------------- * Basic understanding of financial statements (balance sheet, income statement, cash flow) * Familiarity with [finding companies](https://edgartools.readthedocs.io/en/latest/guides/finding-companies/) and [searching filings](https://edgartools.readthedocs.io/en/latest/guides/searching-filings/) Quick Start: Single Period Statements ------------------------------------- ### Get Latest Financial Statements The fastest way to get financial statements is using the Company.financials property: `from edgar import Company # Get Apple's latest financials company = Company("AAPL") financials = company.get_financials() # Access individual statements balance_sheet = financials.balance_sheet income_statement = financials.income_statement() cash_flow = financials.cashflow_statement()` ### Alternative: From Specific Filing For more control, extract statements from a specific filing: `from edgar import Company # Get a specific filing company = Company("AAPL") filing = company.get_filings(form="10-K").latest() # Parse XBRL data xbrl = filing.xbrl() # Access statements through the user-friendly API statements = xbrl.statements # Display financial statements balance_sheet = statements.balance_sheet() income_statement = statements.income_statement() cash_flow = statements.cashflow_statement() print(balance_sheet) # Rich formatted output` ### Enhanced Dimensional Display EdgarTools now automatically surfaces rich dimensional segment data in financial statements when available: `# Get Microsoft's income statement - now shows product/service breakdowns company = Company("MSFT") xbrl = company.get_filings(form="10-K").latest().xbrl() income_stmt = xbrl.statements.income_statement() print(income_stmt) # Output shows both summary revenue AND detailed breakdowns: # - Product revenue: $63.9B # - Service revenue: $217.8B # - Business segment details (LinkedIn: $17.8B, Gaming: $23.5B, etc.)` **Control Dimensional Display:** `# Default behavior - includes dimensional segment data df_enhanced = income_stmt.to_dataframe() # 48 rows for Microsoft print(f"Enhanced view: {len(df_enhanced)} rows") # Standard view - face presentation only df_standard = income_stmt.to_dataframe(view="standard") # 21 rows print(f"Standard view: {len(df_standard)} rows") # Summary view - non-dimensional totals only df_summary = income_stmt.to_dataframe(view="summary") print(f"Summary view: {len(df_summary)} rows")` **What Gets Enhanced:** - Product/service revenue breakdowns - Geographic segment data \- Business unit financial details - Any ProductOrServiceAxis dimensional facts This enhancement works automatically across companies that provide segment data in their XBRL filings, including Microsoft, Apple, Amazon, Google, and many others. Understanding Statement Hierarchy --------------------------------- Every financial statement has a tree structure — revenue breaks down into product vs service revenue, operating expenses break down into R&D, SG&A, etc. When you call `to_dataframe()`, EdgarTools includes hierarchy columns that let you navigate these relationships programmatically. ### Hierarchy Columns The DataFrame returned by `to_dataframe()` includes these columns for understanding structure: | Column | Description | Example | | --- | --- | --- | | `level` | Nesting depth (0=root, 1=section, 2=line item, 3=sub-item) | `2` | | `abstract` | True for section headers, False for data rows | `True` | | `parent_concept` | Calculation tree parent — the metric this rolls up to | `us-gaap_Revenue` | | `parent_abstract_concept` | Presentation tree parent — the section header above | `us-gaap_RevenueAbstract` | ### Example: Apple Revenue Breakdown `from edgar import Company company = Company("AAPL") filing = company.get_filings(form="10-K").latest() xbrl = filing.xbrl() income = xbrl.statements.income_statement() df = income.to_dataframe() # Show hierarchy for the first 10 rows print(df[['label', 'level', 'parent_concept', 'parent_abstract_concept']].head(10))` Output shows the tree structure: `label level parent_concept parent_abstract_concept 0 Net sales: 1 None us-gaap_RevenueFromCont... 1 Products 2 us-gaap_Revenue... us-gaap_RevenueFromCont... 2 Services 2 us-gaap_Revenue... us-gaap_RevenueFromCont... 3 Total net sales 1 None us-gaap_RevenueFromCont... 4 Cost of sales: 1 None us-gaap_CostOfGoodsSold... 5 Products 2 us-gaap_CostOf... us-gaap_CostOfGoodsSold... 6 Services 2 us-gaap_CostOf... us-gaap_CostOfGoodsSold...` ### Building a Parent-Child Tree Use the `level` column to reconstruct the hierarchy: `# Get all non-abstract rows with their hierarchy info data_rows = df[~df['abstract']] hierarchy = data_rows[['label', 'level', 'parent_concept', 'parent_abstract_concept']].copy() # Show indented tree for _, row in hierarchy.iterrows(): indent = ' ' * row['level'] print(f"{indent}{row['label']}")` ### Extracting Revenue Segments To get just the revenue breakdown with parent-child relationships: `# Find revenue-related rows using parent_concept revenue_concept = 'us-gaap_RevenueFromContractWithCustomerExcludingAssessedTax' # All rows whose parent is Revenue (these are the segments) segments = df[df['parent_concept'] == revenue_concept] print(segments[['label', 'level']].to_string(index=False))` ### When to Use `query()` vs `Statement` | Goal | Use | Why | | --- | --- | --- | | Hierarchical data with parent-child | `statement.to_dataframe()` | Preserves presentation tree structure | | Flat fact extraction with filters | `xbrl.query()` | Efficient for specific concept lookups | | Revenue segment tree | `statement.to_dataframe()` | Has `level`, `parent_concept` columns | | Specific dimensional breakdowns | `statement.to_dataframe(view="detailed")` | Includes all dimensional data | | Cross-filing fact comparison | `xbrl.query()` | Works across periods and filings | ### Controlling Detail Level with `view` The `view` parameter controls how much dimensional data appears: `# Standard: face presentation only (what you see in SEC Viewer) df_standard = income.to_dataframe(view="standard") # Detailed: includes all dimensional breakdowns (segments, geography, etc.) df_detailed = income.to_dataframe(view="detailed") # Summary: non-dimensional totals only df_summary = income.to_dataframe(view="summary")` See the [Dimension Handling Guide](https://edgartools.readthedocs.io/en/latest/xbrl/concepts/dimension-handling/) for more on controlling dimensional data. * * * Standardized Financial Data Access ---------------------------------- EdgarTools automatically standardizes XBRL data across companies, mapping ~2,000 different XBRL tags to 95 consistent concepts. This means you can compare Apple's revenue with Tesla's revenue using the same API, even though they use different underlying XBRL concepts. **Why this matters:** - Companies use different XBRL tags for the same concept (e.g., "Revenues", "RevenueFromContractWithCustomer", "SalesRevenueNet") - EdgarTools normalizes these to standard concepts like "Revenue" - Cross-company analysis becomes trivial For the complete list of 95 standard concepts and their mappings, see [Standardization Concepts Reference](https://edgartools.readthedocs.io/en/latest/xbrl/concepts/standardization/) . ### Simple Metric Extraction The easiest way to get key financial metrics is using the standardized accessor methods: `from edgar import Company # Get a company's financials company = Company("AAPL") financials = company.get_financials() # Extract key metrics directly - these work across all companies! revenue = financials.get_revenue() net_income = financials.get_net_income() total_assets = financials.get_total_assets() print(f"Revenue: ${revenue:,.0f}") print(f"Net Income: ${net_income:,.0f}") print(f"Total Assets: ${total_assets:,.0f}")` This simple API works consistently across all companies, regardless of their custom XBRL concepts! ### Available Standardized Methods All methods support historical data via the `period_offset` parameter: `# Income Statement Metrics revenue_current = financials.get_revenue() # Current period revenue_previous = financials.get_revenue(1) # Previous period net_income = financials.get_net_income() # Balance Sheet Metrics total_assets = financials.get_total_assets() total_liabilities = financials.get_total_liabilities() stockholders_equity = financials.get_stockholders_equity() current_assets = financials.get_current_assets() current_liabilities = financials.get_current_liabilities() # Cash Flow Metrics operating_cash_flow = financials.get_operating_cash_flow() capital_expenditures = financials.get_capital_expenditures() free_cash_flow = financials.get_free_cash_flow() # Calculated automatically` ### Comprehensive Financial Analysis Get all key metrics at once with automatic ratio calculations: `# Get comprehensive metrics dictionary metrics = financials.get_financial_metrics() # All the standard metrics are available print(f"Revenue: ${metrics['revenue']:,.0f}") print(f"Net Income: ${metrics['net_income']:,.0f}") print(f"Total Assets: ${metrics['total_assets']:,.0f}") # Plus calculated ratios print(f"Current Ratio: {metrics['current_ratio']:.2f}") print(f"Debt to Assets: {metrics['debt_to_assets']:.2f}") print(f"Free Cash Flow: ${metrics['free_cash_flow']:,.0f}")` ### Cross-Company Analysis Made Simple Now comparing multiple companies is trivial: `companies = ["AAPL", "MSFT", "GOOGL", "AMZN", "META"] print("Company\t\tRevenue\t\tNet Income\tTotal Assets") print("-" * 60) for ticker in companies: company = Company(ticker) financials = company.get_financials() if financials: revenue = financials.get_revenue() net_income = financials.get_net_income() total_assets = financials.get_total_assets() print(f"{ticker}\t\t${revenue/1e9:.1f}B\t\t${net_income/1e9:.1f}B\t\t${total_assets/1e9:.1f}B")` ### Tesla Custom Concepts - No Problem! The standardized methods automatically handle companies with custom concepts like Tesla: `# Works even with companies that use non-standard XBRL concepts tesla = Company("TSLA") tsla_financials = tesla.get_financials() # These work despite Tesla's custom concepts tsla_revenue = tsla_financials.get_revenue() tsla_net_income = tsla_financials.get_net_income() print(f"Tesla Revenue: ${tsla_revenue:,.0f}") print(f"Tesla Net Income: ${tsla_net_income:,.0f}")` ### Growth Analysis with Historical Data Calculate growth rates using the `period_offset` parameter: `# Get current and previous year data current_revenue = financials.get_revenue(0) # Current period previous_revenue = financials.get_revenue(1) # Previous period if current_revenue and previous_revenue: growth_rate = (current_revenue - previous_revenue) / previous_revenue * 100 print(f"Revenue Growth: {growth_rate:.1f}%") # Same pattern works for any metric current_ni = financials.get_net_income(0) previous_ni = financials.get_net_income(1) if current_ni and previous_ni: ni_growth = (current_ni - previous_ni) / previous_ni * 100 print(f"Net Income Growth: {ni_growth:.1f}%")` Multi-Period Analysis --------------------- ### Method 1: Using MultiFinancials Get financials across multiple years for trend analysis: `from edgar import Company, MultiFinancials # Get multiple years of 10-K filings company = Company("AAPL") filings = company.get_filings(form="10-K").head(3) # Last 3 annual reports # Create multi-period financials multi_financials = MultiFinancials.extract(filings) # Access statements spanning multiple years balance_sheet = multi_financials.balance_sheet() income_statement = multi_financials.income_statement() cash_flow = multi_financials.cashflow_statement() # Use view="detailed" to include dimensional breakdowns (e.g., cost by segment) income_detailed = multi_financials.income_statement(view="detailed") print("Multi-Year Income Statement:") print(income_statement)` ### Method 2: Using XBRL Stitching For more advanced multi-period analysis with intelligent period matching: `from edgar import Company from edgar.xbrl import XBRLS # Get multiple filings for trend analysis company = Company("AAPL") filings = company.get_filings(form="10-K").head(3) # Create stitched view across multiple filings xbrls = XBRLS.from_filings(filings) # Access stitched statements stitched_statements = xbrls.statements # Display multi-period statements with intelligent period selection income_trend = stitched_statements.income_statement() balance_sheet_trend = stitched_statements.balance_sheet() cashflow_trend = stitched_statements.cashflow_statement() print("Three-Year Revenue Trend:") revenue_trend = income_trend.to_dataframe() revenue_row = revenue_trend.loc[revenue_trend['label'] == 'Revenue'] print(revenue_row)` **Dimensional Data in Stitching:** By default, stitching uses traditional statement structures for performance and compatibility. Use the `view` parameter to control dimensional data: `# Default stitching - standard face presentation for multi-period consistency income_stmt = stitched_statements.income_statement() # Clean, focused view # Include dimensional breakdowns (e.g., cost of operations by segment) income_stmt_detailed = stitched_statements.income_statement(view="detailed") # Summary view - non-dimensional totals only income_stmt_summary = stitched_statements.income_statement(view="summary")` **When to Use Each View:** * **`"standard"`** (default): Best for trend analysis, ratios, and cross-period comparisons * **`"detailed"`**: Use when you need segment data across periods (e.g., cost breakdowns by product line) * **`"summary"`**: Quick overview of main line items only Working with Individual Statements ---------------------------------- ### Balance Sheet Analysis `# Get balance sheet balance_sheet = statements.balance_sheet() # Convert to DataFrame for analysis bs_df = balance_sheet.to_dataframe() # Extract key balance sheet items total_assets = bs_df[bs_df['label'] == 'Total Assets'] total_liabilities = bs_df[bs_df['label'] == 'Total Liabilities'] shareholders_equity = bs_df[bs_df['label'] == "Total Stockholders' Equity"] print("Balance Sheet Summary:") print(f"Total Assets: ${total_assets.iloc[0, -1]/1e9:.1f}B") print(f"Total Liabilities: ${total_liabilities.iloc[0, -1]/1e9:.1f}B") print(f"Shareholders' Equity: ${shareholders_equity.iloc[0, -1]/1e9:.1f}B") # Calculate debt-to-equity ratio debt_to_equity = total_liabilities.iloc[0, -1] / shareholders_equity.iloc[0, -1] print(f"Debt-to-Equity Ratio: {debt_to_equity:.2f}")` ### Income Statement Analysis `# Get income statement income_statement = statements.income_statement() # Convert to DataFrame is_df = income_statement.to_dataframe() # Extract key income statement items revenue = is_df[is_df['label'] == 'Revenue'] gross_profit = is_df[is_df['label'] == 'Gross Profit'] operating_income = is_df[is_df['label'] == 'Operating Income'] net_income = is_df[is_df['label'] == 'Net Income'] print("Income Statement Analysis:") print(f"Revenue: ${revenue.iloc[0, -1]/1e9:.1f}B") print(f"Gross Profit: ${gross_profit.iloc[0, -1]/1e9:.1f}B") print(f"Operating Income: ${operating_income.iloc[0, -1]/1e9:.1f}B") print(f"Net Income: ${net_income.iloc[0, -1]/1e9:.1f}B") # Calculate margins gross_margin = (gross_profit.iloc[0, -1] / revenue.iloc[0, -1]) * 100 operating_margin = (operating_income.iloc[0, -1] / revenue.iloc[0, -1]) * 100 net_margin = (net_income.iloc[0, -1] / revenue.iloc[0, -1]) * 100 print(f"\nMargin Analysis:") print(f"Gross Margin: {gross_margin:.1f}%") print(f"Operating Margin: {operating_margin:.1f}%") print(f"Net Margin: {net_margin:.1f}%")` ### Cash Flow Analysis `# Get cash flow statement cash_flow = statements.cashflow_statement() # Convert to DataFrame cf_df = cash_flow.to_dataframe() # Extract cash flow components operating_cf = cf_df[cf_df['label'] == 'Net Cash from Operating Activities'] investing_cf = cf_df[cf_df['label'] == 'Net Cash from Investing Activities'] financing_cf = cf_df[cf_df['label'] == 'Net Cash from Financing Activities'] print("Cash Flow Analysis:") print(f"Operating Cash Flow: ${operating_cf.iloc[0, -1]/1e9:.1f}B") print(f"Investing Cash Flow: ${investing_cf.iloc[0, -1]/1e9:.1f}B") print(f"Financing Cash Flow: ${financing_cf.iloc[0, -1]/1e9:.1f}B") # Calculate free cash flow (Operating CF - Capital Expenditures) capex = cf_df[cf_df['label'].str.contains('Capital Expenditures', case=False, na=False)] if not capex.empty: free_cash_flow = operating_cf.iloc[0, -1] + capex.iloc[0, -1] # CapEx is usually negative print(f"Free Cash Flow: ${free_cash_flow/1e9:.1f}B")` Advanced Statement Customization -------------------------------- ### Period Views and Formatting `# Get available period views for income statement period_views = statements.get_period_views("IncomeStatement") print("Available period views:") for view in period_views: print(f"- {view['name']}: {view['description']}") # Render with specific period view annual_comparison = statements.income_statement(period_view="Annual Comparison") quarterly_comparison = statements.income_statement(period_view="Quarterly Comparison") # Show full date ranges for duration periods income_with_dates = statements.income_statement(show_date_range=True) print("Income Statement with Date Ranges:") print(income_with_dates)` ### Standardized vs Company-Specific Labels When using stitched statements (multi-period analysis), you can control label standardization: `from edgar import Company from edgar.xbrl import XBRLS # Get multiple filings for stitched analysis company = Company("AAPL") filings = company.get_filings(form="10-K").head(3) xbrls = XBRLS.from_filings(filings) stitched = xbrls.statements # Get income statement with standard_concept metadata (default) income = stitched.income_statement(standard=True) # Labels always show original company presentation # Use standard_concept for cross-company analysis df = income.to_dataframe() print("Labels with Standard Concept Mapping:") print(df[['label', 'standard_concept']].head(10)) # Aggregate by standard concept for comparison standardized = df.groupby('standard_concept')[df.columns[2:4]].sum() print("\nAggregated by Standard Concept:") print(standardized.head(10))` > **Note**: The `standard=True` parameter adds `standard_concept` metadata for cross-company analysis. Labels always preserve the company's original presentation. Cross-Company Analysis ---------------------- ### Compare Multiple Companies (Updated with New API!) `import pandas as pd def get_key_metrics(ticker): """Extract key financial metrics for a company using new standardized methods.""" try: company = Company(ticker) financials = company.get_financials() if not financials: return None # Use the new standardized accessor methods - much simpler! return { 'ticker': ticker, 'revenue': financials.get_revenue(), 'net_income': financials.get_net_income(), 'total_assets': financials.get_total_assets(), 'operating_cf': financials.get_operating_cash_flow(), 'free_cf': financials.get_free_cash_flow() } except Exception as e: print(f"Error processing {ticker}: {e}") return None # Analyze multiple companies tech_companies = ['AAPL', 'MSFT', 'GOOGL', 'AMZN', 'META'] metrics = [] for ticker in tech_companies: result = get_key_metrics(ticker) if result: metrics.append(result) # Create comparison DataFrame comparison_df = pd.DataFrame(metrics) # Convert to billions and calculate ratios comparison_df['revenue_b'] = comparison_df['revenue'] / 1e9 comparison_df['net_income_b'] = comparison_df['net_income'] / 1e9 comparison_df['net_margin'] = (comparison_df['net_income'] / comparison_df['revenue']) * 100 comparison_df['roa'] = (comparison_df['net_income'] / comparison_df['total_assets']) * 100 print("Tech Giants Comparison:") print(comparison_df[['ticker', 'revenue_b', 'net_income_b', 'net_margin', 'roa']].round(1))` The new standardized methods make cross-company analysis much more reliable and easier to implement! Notes and Disclosures --------------------- XBRL filings contain notes and disclosure sections beyond the primary financial statements. Access them with convenience methods: `xbrl = filing.xbrl() # Browse all note sections (e.g., accounting policies, segment data) for note in xbrl.notes(): print(note.title) # Browse all disclosure sections (e.g., revenue disaggregation, debt details) for disc in xbrl.disclosures(): print(disc.title)` These return the same statement objects as `xbrl.statements`, filtered to notes and disclosures respectively. Advanced XBRL Features ---------------------- ### Access Raw XBRL Facts `# Access the facts API for detailed XBRL data facts = xbrl.facts # Query facts by concept revenue_facts = facts.query().by_concept('Revenue').to_dataframe() print("Revenue facts across all periods:") print(revenue_facts[['concept', 'label', 'period', 'value']]) # Search for specific concepts earnings_facts = facts.search_facts("Earnings Per Share") print("EPS-related facts:") print(earnings_facts[['concept', 'label', 'value']]) # Get facts by statement type balance_sheet_facts = facts.query().by_statement_type('BalanceSheet').to_dataframe() print(f"Found {len(balance_sheet_facts)} balance sheet facts")` ### Time Series Analysis `# Get time series data for specific concepts revenue_series = facts.time_series('Revenue') net_income_series = facts.time_series('Net Income') print("Revenue Time Series:") print(revenue_series) # Convert to DataFrame for analysis import pandas as pd ts_df = pd.DataFrame({ 'revenue': revenue_series, 'net_income': net_income_series }) # Calculate growth rates ts_df['revenue_growth'] = ts_df['revenue'].pct_change() * 100 ts_df['income_growth'] = ts_df['net_income'].pct_change() * 100 print("Growth Analysis:") print(ts_df[['revenue_growth', 'income_growth']].round(1))` ### Dimensional Analysis `# Query facts by dimensions (if available) segment_facts = facts.query().by_dimension('Segment').to_dataframe() if not segment_facts.empty: print("Segment-specific financial data:") print(segment_facts[['concept', 'label', 'dimension_value', 'value']].head()) # Get facts by geographic dimension geographic_facts = facts.query().by_dimension('Geography').to_dataframe() if not geographic_facts.empty: print("Geographic breakdown:") print(geographic_facts[['concept', 'dimension_value', 'value']].head())` Export and Integration ---------------------- ### Export to Different Formats `# Export statements to various formats income_statement = statements.income_statement() # Export to pandas DataFrame df = income_statement.to_dataframe() # Export to markdown markdown_text = income_statement.render().to_markdown() # Save to CSV df.to_csv('apple_income_statement.csv', index=False) # Save markdown to file with open('apple_income_statement.md', 'w') as f: f.write(markdown_text) print("Statements exported to CSV and Markdown")` ### Integration with Analysis Libraries `import matplotlib.pyplot as plt import seaborn as sns # Get multi-period data filings = company.get_filings(form="10-K").head(5) multi_financials = MultiFinancials.extract(filings) income_df = multi_financials.income.to_dataframe() # Extract revenue data for plotting revenue_data = income_df[income_df['label'] == 'Revenue'].iloc[0, 1:].astype(float) periods = revenue_data.index # Create visualization plt.figure(figsize=(10, 6)) plt.plot(periods, revenue_data / 1e9, marker='o', linewidth=2) plt.title('Apple Revenue Trend (5 Years)') plt.xlabel('Period') plt.ylabel('Revenue (Billions USD)') plt.xticks(rotation=45) plt.grid(True, alpha=0.3) plt.tight_layout() plt.show() # Calculate year-over-year growth revenue_growth = revenue_data.pct_change() * 100 print("Year-over-Year Revenue Growth:") for period, growth in revenue_growth.dropna().items(): print(f"{period}: {growth:.1f}%")` Performance Optimization ------------------------ ### Efficient Multi-Company Analysis `# Efficient batch processing def batch_analyze_companies(tickers, max_workers=5): """Analyze multiple companies efficiently.""" from concurrent.futures import ThreadPoolExecutor def analyze_single(ticker): try: company = Company(ticker) financials = company.financials return { 'ticker': ticker, 'revenue': financials.income.loc['Revenue'].iloc[0], 'assets': financials.balance_sheet.loc['Total Assets'].iloc[0] } except Exception as e: return {'ticker': ticker, 'error': str(e)} with ThreadPoolExecutor(max_workers=max_workers) as executor: results = list(executor.map(analyze_single, tickers)) return [r for r in results if 'error' not in r] # Analyze S&P 100 companies efficiently sp100_sample = ['AAPL', 'MSFT', 'GOOGL', 'AMZN', 'META', 'TSLA', 'NVDA', 'JPM'] results = batch_analyze_companies(sp100_sample) comparison_df = pd.DataFrame(results) print("Batch Analysis Results:") print(comparison_df.head())` ### Caching for Repeated Analysis `# Cache XBRL data for repeated use company = Company("AAPL") filing = company.get_filings(form="10-K").latest() # Parse once, use multiple times xbrl = filing.xbrl() # Perform different analyses on same data balance_sheet = xbrl.statements.balance_sheet() income_statement = xbrl.statements.income_statement() cash_flow = xbrl.statements.cashflow_statement() # Access facts for custom queries facts = xbrl.facts revenue_facts = facts.query().by_concept('Revenue').to_dataframe() margin_facts = facts.search_facts("margin")` Common Patterns and Best Practices ---------------------------------- ### Robust Financial Metric Extraction `def safe_extract_metric(statement_df, label, column=-1, default=None): """Safely extract a metric from financial statement DataFrame.""" try: rows = statement_df[statement_df['label'].str.contains(label, case=False, na=False)] if not rows.empty: return rows.iloc[0, column] return default except Exception: return default # Use for robust metric extraction income_df = statements.income_statement().to_dataframe() revenue = safe_extract_metric(income_df, 'Revenue') net_income = safe_extract_metric(income_df, 'Net Income') operating_income = safe_extract_metric(income_df, 'Operating Income') if revenue and net_income: net_margin = (net_income / revenue) * 100 print(f"Net Margin: {net_margin:.1f}%")` ### Handle Missing or Inconsistent Data `def get_financial_metrics(company_ticker): """Get financial metrics with error handling.""" try: company = Company(company_ticker) financials = company.financials metrics = {} # Try to get income statement metrics try: income = financials.income metrics['revenue'] = income.loc['Revenue'].iloc[0] if 'Revenue' in income.index else None metrics['net_income'] = income.loc['Net Income'].iloc[0] if 'Net Income' in income.index else None except Exception as e: print(f"Income statement error for {company_ticker}: {e}") # Try to get balance sheet metrics try: balance_sheet = financials.balance_sheet metrics['total_assets'] = balance_sheet.loc['Total Assets'].iloc[0] if 'Total Assets' in balance_sheet.index else None except Exception as e: print(f"Balance sheet error for {company_ticker}: {e}") return metrics except Exception as e: print(f"Company error for {company_ticker}: {e}") return {} # Test with various companies test_companies = ['AAPL', 'INVALID_TICKER', 'MSFT'] for ticker in test_companies: metrics = get_financial_metrics(ticker) if metrics: print(f"{ticker}: {metrics}")` Troubleshooting Common Issues ----------------------------- ### Statement Not Available `# Check what statements are available try: statements = xbrl.statements available_statements = statements.available_statements() print(f"Available statements: {available_statements}") # Try alternative statement access if 'IncomeStatement' in available_statements: income = statements.income_statement() elif 'ComprehensiveIncome' in available_statements: income = statements['ComprehensiveIncome'] else: print("No income statement available") except Exception as e: print(f"Error accessing statements: {e}")` ### Period Selection Issues `# Check available periods reporting_periods = xbrl.reporting_periods print("Available reporting periods:") for period in reporting_periods[:5]: # Show first 5 print(f"- {period['date']} ({period['type']}): {period.get('duration', 'N/A')} days") # Handle quarterly vs annual periods if any(p.get('duration', 0) < 120 for p in reporting_periods): print("Quarterly periods detected") quarterly_income = statements.income_statement(period_view="Quarterly Comparison") else: print("Annual periods only") annual_income = statements.income_statement(period_view="Annual Comparison")` Next Steps ---------- Now that you can extract financial statements, explore these advanced topics: * **[XBRL Documentation Hub](https://edgartools.readthedocs.io/en/latest/xbrl/) ** - Central navigation for all XBRL documentation * **[Multi-Period Analysis](https://edgartools.readthedocs.io/en/latest/xbrl/guides/multi-period-analysis/) ** - Compare financials across multiple years * **[Choosing the Right API](https://edgartools.readthedocs.io/en/latest/xbrl/getting-started/choosing-the-right-api/) ** - Decision guide for which API to use * **[Dimension Handling Guide](https://edgartools.readthedocs.io/en/latest/xbrl/concepts/dimension-handling/) ** - Understanding dimensional data (segments, breakdowns) * **[Standardization Concepts](https://edgartools.readthedocs.io/en/latest/xbrl/concepts/standardization/) ** - 95 standard concepts for cross-company comparison Related Documentation --------------------- * **[Getting XBRL from Filings](https://edgartools.readthedocs.io/en/latest/getting-xbrl/) ** - Original XBRL documentation * **[Company Financials](https://edgartools.readthedocs.io/en/latest/guides/company-financials.md) ** - Company financials API * **[XBRL API Reference](https://edgartools.readthedocs.io/en/latest/api/xbrl/) ** - Complete XBRL class documentation * **[StatementType Quick Reference](https://edgartools.readthedocs.io/en/latest/StatementType-Quick-Reference/) ** - Statement type enums and API comparison Back to top --- # Current Events (8-K) - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/eightk-filings/#8-k-current-reports-parse-sec-corporate-events-and-earnings-with-python) 8-K Current Reports: Parse SEC Corporate Events and Earnings with Python ======================================================================== Companies file 8-K current reports within four business days of material events -- acquisitions, executive changes, earnings releases, bankruptcy. EdgarTools parses these filings into structured Python objects so you can access the event items, press releases, and financial tables. `from edgar import * filing = get_filings(form="8-K").latest() eight_k = filing.obj() eight_k` ![8-K current report parsed with Python edgartools](https://edgartools.readthedocs.io/en/stable/images/eightk.webp) Three lines to get a parsed 8-K with company info, filing date, items disclosed, and exhibits. > **[See today's 8-K filings on edgar.tools — with AI-classified material event types →](https://app.edgar.tools/filings?utm_source=edgartools-docs&utm_medium=see-live&utm_content=eightk) > ** * * * Read Item Content ----------------- Every 8-K discloses one or more numbered items (1.01 through 9.01). The `items` property lists what was disclosed: `eight_k.items # ['Item 2.02', 'Item 9.01']` Access the text of any item by number: `# Works with or without "Item" prefix content = eight_k['2.02'] content = eight_k['Item 2.02'] print(content)` Common items: | Item | What it reports | | --- | --- | | **1.01** | Material agreements | | **2.02** | Earnings and financial condition | | **5.02** | Director or officer changes | | **8.01** | Other events | | **9.01** | Financial statements and exhibits | The complete mapping is in `eight_k.structure`. * * * Access Press Releases --------------------- Most 8-Ks attach press releases as EX-99 exhibits: `if eight_k.has_press_release: releases = eight_k.press_releases pr = releases[0] # Get content in different formats pr.text() # Plain text pr.html() # HTML pr.to_markdown() # Markdown pr.open() # Open in browser` Press releases are indexed by position. `press_releases[0]` is the first release, `press_releases[1]` is the second. * * * Extract Earnings and Financial Statements from 8-K Filings ---------------------------------------------------------- When a company reports quarterly earnings, the numbers first appear in an 8-K -- weeks before the formal 10-Q. The company files **Item 2.02** ("Results of Operations and Financial Condition") and attaches the earnings press release as an EX-99.1 exhibit. That exhibit contains HTML tables with income statements, balance sheets, and cash flows. EdgarTools detects this pattern and parses the tables automatically. `from edgar import Company aapl = Company("AAPL") filing = aapl.get_filings(form="8-K").latest() eight_k = filing.obj() eight_k.has_earnings # True if Item 2.02 + parseable EX-99.1 exhibit` ### Get Financial Statements `if eight_k.has_earnings: # Direct access to parsed statements eight_k.income_statement # FinancialTable or None eight_k.balance_sheet # FinancialTable or None eight_k.cash_flow_statement # FinancialTable or None # Safe accessors -- always return a DataFrame (empty if missing) df = eight_k.get_income_statement() df = eight_k.get_balance_sheet() df = eight_k.get_cash_flow_statement()` ### Work with the Data Each statement is a `FinancialTable` with the parsed data and detected scale: `income = eight_k.income_statement income.dataframe # Parsed data as DataFrame income.scaled_dataframe # Values multiplied by detected scale income.scale # Scale enum (UNITS, THOUSANDS, MILLIONS, BILLIONS) income.title # Table title if detected income.to_html() # HTML export for web apps income.to_json() # JSON export for APIs` Press releases often report values "in millions" or "in thousands." EdgarTools detects the scale from the document text. Use `scaled_dataframe` to get actual dollar amounts, or check `scale` to apply your own multiplier. ### Access All Parsed Tables The `earnings` property returns the full `EarningsRelease` object, which may contain more than just the three core statements: `earnings = eight_k.earnings earnings.financial_tables # All parsed financial tables earnings.segment_data # Business segment breakdown earnings.eps_reconciliation # GAAP to Non-GAAP EPS reconciliation earnings.guidance # Forward guidance table earnings.detected_scale # Document-level scale factor` Not every press release includes all of these -- `segment_data`, `eps_reconciliation`, and `guidance` return `None` when the company doesn't report them. * * * Work with Exhibits ------------------ All 8-K attachments (press releases, financial statements, material agreements): `exhibits = filing.exhibits for ex in exhibits: print(f"{ex.document_type}: {ex.description}") # Access specific exhibit ex_99 = exhibits[0] content = ex_99.download()` Exhibits are indexed by position. The `document_type` shows what kind of exhibit it is (EX-99.1, EX-10.1, etc.). * * * Common Analysis Patterns ------------------------ ### Find all earnings releases in a quarter `from edgar import get_filings filings = get_filings( form="8-K", date="2024-01-01:2024-03-31" ) for filing in filings[:20]: eight_k = filing.obj() if eight_k.has_earnings: print(f"{filing.company}: {filing.filing_date}")` ### Extract all financial tables `if eight_k.has_earnings: for table in eight_k.earnings.financial_tables: print(f"{table.statement_type.value}: {table.dataframe.shape}")` ### Check for specific events `# Director changes if 'Item 5.02' in eight_k.items: print(eight_k['5.02']) # Material agreements if 'Item 1.01' in eight_k.items: print(eight_k['1.01']) # Stock split announcements (Item 8.01 or 5.03) if 'Item 8.01' in eight_k.items or 'Item 5.03' in eight_k.items: content = eight_k.get('8.01') or eight_k.get('5.03') or '' if 'split' in content.lower(): print(f"Stock split announced: {filing.filing_date}")` See this on edgar.tools The code above detects 8-K event types by checking item numbers manually. **edgar.tools** classifies material events automatically using LLMs — earnings, acquisitions, executive changes, and more — across every 8-K as it's filed. * **[Watch 8-K filings arrive in real time with event classification →](https://app.edgar.tools/filings?utm_source=edgartools-docs&utm_medium=see-live&utm_content=eightk) ** * **[See Apple's recent material events →](https://app.edgar.tools/companies/AAPL?utm_source=edgartools-docs&utm_medium=see-live&utm_content=eightk) ** Includes AI-generated summaries, business descriptions, and cross-filing linkages. Free tier available. [Pricing →](https://app.edgar.tools/pricing?utm_source=edgartools-docs&utm_medium=see-live&utm_content=eightk) * * * Metadata Quick Reference ------------------------ | Property | Returns | Example | | --- | --- | --- | | `company` | Company name | `"Apple Inc."` | | `form` | Form type | `"8-K"` | | `filing_date` | Date filed with SEC | `"2024-02-01"` | | `period_of_report` | Report date | `"2024-01-31"` | | `date_of_report` | Formatted report date | `"January 31, 2024"` | | `items` | List of disclosed items | `['Item 2.02', 'Item 9.01']` | | `has_press_release` | Has EX-99 press release? | `True` | | `has_earnings` | Has parseable earnings data? | `True` | * * * Methods Quick Reference ----------------------- | Call | Returns | What it does | | --- | --- | --- | | `eight_k['2.02']` | `str` | Get item content by number | | `eight_k.press_releases` | `PressReleases` | Collection of press release exhibits | | `eight_k.earnings` | `EarningsRelease` | Parsed earnings tables from EX-99.1 | | `eight_k.income_statement` | `FinancialTable` | Income statement or None | | `eight_k.balance_sheet` | `FinancialTable` | Balance sheet or None | | `eight_k.cash_flow_statement` | `FinancialTable` | Cash flow statement or None | | `eight_k.get_income_statement()` | `DataFrame` | Income statement (empty DataFrame if missing) | | `eight_k.get_balance_sheet()` | `DataFrame` | Balance sheet (empty DataFrame if missing) | | `eight_k.get_cash_flow_statement()` | `DataFrame` | Cash flow (empty DataFrame if missing) | * * * Things to Know -------------- **Item detection is multi-tier.** EdgarTools uses document parser first (95% accuracy), falls back to text extraction for legacy SGML filings (1999-2001). **Earnings parsing requires EX-99.1.** The `has_earnings` property returns True only if Item 2.02 is present AND an EX-99.1 exhibit contains parseable tables. Some earnings 8-Ks only have narrative text. **Scale matters.** Financial tables include scale detection (thousands, millions, billions). Use `scaled_dataframe` to get values with scale applied, or check `table.scale` to apply manually. **Not all 8-Ks have XBRL.** 8-K filings typically contain only DEI (Document and Entity Information) XBRL metadata. Actual financial data is in HTML tables within EX-99 exhibits. **Press releases use pattern matching.** EdgarTools looks for EX-99, EX-99.1, EX-99.01 exhibits or exhibits with "RELEASE" in the description. Some companies use non-standard exhibit numbering. * * * Related ------- * [Stock Splits & EPS Normalization](https://edgartools.readthedocs.io/en/stable/guides/stock-splits-eps-normalization/) -- detect split announcements and normalize metrics * [Working with Filings](https://edgartools.readthedocs.io/en/stable/guides/working-with-filing/) -- general filing access patterns * [10-K Annual Reports](https://edgartools.readthedocs.io/en/stable/eightk-filings/tenk-filings.md) -- annual report parsing * [10-Q Quarterly Reports](https://edgartools.readthedocs.io/en/stable/eightk-filings/tenq-filings.md) -- quarterly report parsing Back to top --- # Advanced Search - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/advanced-search/#advanced-ranking-search) Advanced Ranking Search ======================= EdgarTools provides advanced search capabilities with BM25-based ranking and semantic structure awareness. This is designed specifically for financial documents to help you find the most relevant information quickly. Overview -------- The ranking search system offers: * **Three ranking algorithms**: BM25 (text-focused), Hybrid (text + structure), and Semantic (structure-focused) * **Intelligent caching**: 10x+ faster repeated searches with automatic index caching * **Structure-aware boosting**: Prioritizes headings, cross-references, and gateway content * **Agent-friendly results**: Full section context for investigation and navigation * **Performance optimized**: Instant results from cache, minimal memory overhead Quick Start ----------- `from edgar import Company from edgar.documents.search import DocumentSearch # Get a filing company = Company("AAPL") filing = company.get_filings(form="10-K").latest(1) # Parse HTML into structured Document document = filing.parse() # Create search interface searcher = DocumentSearch(document) # Search with ranking results = searcher.ranked_search( query="revenue growth", algorithm="hybrid", top_k=5 ) # Access results for result in results: print(f"Score: {result.score:.3f}") print(f"Section: {result.section}") print(f"Snippet: {result.snippet}")` About `filing.parse()` The `parse()` method parses the filing's HTML into a structured `Document` object with a node tree that `DocumentSearch` can index. This is different from `filing.document` which returns an `Attachment` (file metadata). Other useful Filing methods: * `filing.html()` - Returns raw HTML string * `filing.text()` - Returns plain text extraction * `filing.xbrl()` - Returns parsed XBRL data * `filing.parse()` - Returns structured Document for advanced operations Ranking Algorithms ------------------ ### BM25 (Best for Exact Term Matching) BM25 is a probabilistic retrieval function that ranks documents based on term frequency and inverse document frequency. It's excellent for finding exact matches of financial terms and concepts. `results = searcher.ranked_search( query="operating expenses depreciation", algorithm="bm25", top_k=10 )` **Best for:** - Finding specific financial terms - Exact phrase matching - Traditional keyword search **Parameters:** - `k1` (default: 1.5): Controls term frequency saturation - `b` (default: 0.75): Controls document length normalization ### Hybrid (Recommended for Most Use Cases) Hybrid combines BM25 text matching with semantic structure boosting. It understands document structure and boosts: * **Headings and section markers** (e.g., "Item 1A - Risk Factors") * **Cross-references** (e.g., "See Item 7 for discussion") * **Gateway content** (summaries, overviews, introductions) * **Tables and XBRL data** (structured financial information) `results = searcher.ranked_search( query="cybersecurity risks", algorithm="hybrid", top_k=5 )` **Best for:** - General-purpose search - Finding gateway content for investigation - Balancing exact matches with structural importance - Agent/LLM workflows **Weights (customizable):** - `bm25_weight` (default: 0.8): Weight for text matching - `semantic_weight` (default: 0.2): Weight for structure boosting ### Semantic (Best for Structure Navigation) Semantic ranking prioritizes document structure without text matching. It finds structurally important sections regardless of query terms. `results = searcher.ranked_search( query="business overview", algorithm="semantic", top_k=5 )` **Best for:** - Understanding document organization - Finding section boundaries - Structural navigation - Overview and summary content Advanced Search Options ----------------------- ### Section-Specific Search Limit search to specific sections: `results = searcher.ranked_search( query="supply chain risks", in_section="Risk Factors", top_k=5 )` ### Section Boosting Give higher weight to matches in certain sections: `results = searcher.ranked_search( query="revenue recognition", algorithm="hybrid", boost_sections=["MD&A", "Critical Accounting Policies"], top_k=5 )` ### Node Type Filtering Search only specific node types: `from edgar.documents.types import NodeType results = searcher.ranked_search( query="financial data", node_types=[NodeType.TABLE, NodeType.XBRL], top_k=5 )` Working with Results -------------------- Each result contains: `result.score # Relevance score (higher = more relevant) result.snippet # Short text snippet (first 200 chars) result.section # Section name (e.g., "Risk Factors") result.node # Original document node result.context # Full text context (up to 500 chars)` ### Accessing Full Context For agent workflows, results include full section access: `results = searcher.ranked_search("AI strategy", algorithm="hybrid") for result in results: # Access full section for investigation if hasattr(result, '_section_obj') and result._section_obj: section = result._section_obj full_text = section.text() # Navigate section structure for child in section.children: # Process subsections pass` Performance and Caching ----------------------- ### How Caching Works EdgarTools automatically caches search indices for fast repeated searches: 1. **Instance cache**: Stores engines for same DocumentSearch session 2. **Global cache**: Stores indices across documents (memory + disk) 3. **LRU eviction**: Automatically manages memory (default: 10 cached indices) 4. **TTL expiration**: Automatic cleanup after 24 hours ### Cache Performance Typical speedup: `import time # First search (cold cache) - builds index start = time.perf_counter() results1 = searcher.ranked_search("revenue", algorithm="bm25") cold_time = time.perf_counter() - start # ~0.5s # Second search (warm cache) - uses cached index start = time.perf_counter() results2 = searcher.ranked_search("revenue", algorithm="bm25") warm_time = time.perf_counter() - start # ~0.05s # 10x faster!` ### Cache Statistics Monitor cache performance: `stats = searcher.get_cache_stats() print(f"Cache hits: {stats['global_cache_stats']['cache_hits']}") print(f"Cache misses: {stats['global_cache_stats']['cache_misses']}") print(f"Hit rate: {stats['global_cache_stats']['hit_rate']:.1%}") print(f"Memory usage: {stats['global_cache_stats']['memory_size_mb']:.2f} MB")` ### Cache Management `# Clear instance cache only searcher.clear_cache(memory_only=True) # Clear all caches (memory + disk) searcher.clear_cache(memory_only=False) # Disable caching (for testing) searcher = DocumentSearch(document, use_cache=False)` ### Custom Cache Configuration `from edgar.documents.ranking.cache import SearchIndexCache, set_search_cache # Create custom cache cache = SearchIndexCache( memory_cache_size=20, # Store 20 indices in memory disk_cache_enabled=True, # Enable disk persistence ttl_hours=48 # Keep cached for 48 hours ) # Set as global cache set_search_cache(cache)` Best Practices -------------- ### Choosing the Right Algorithm | Use Case | Algorithm | Why | | --- | --- | --- | | Finding specific terms | BM25 | Exact text matching | | General document search | Hybrid | Balance text + structure | | Understanding document structure | Semantic | Pure structure focus | | Agent/LLM workflows | Hybrid | Finds gateway content | | Financial term lookup | BM25 | Best for exact matches | ### Performance Tips 1. **Use caching** (enabled by default) for repeated searches 2. **Use Hybrid algorithm** for most use cases (best results) 3. **Filter by section** to reduce search space 4. **Limit top\_k** to needed results (default: 10) 5. **Monitor cache stats** to optimize cache size ### Agent Workflows For AI agents investigating documents: `# Step 1: Find relevant sections results = searcher.ranked_search( query="climate risk disclosures", algorithm="hybrid", top_k=3 ) # Step 2: Investigate full sections for result in results: if result._section_obj: section = result._section_obj # Read full section full_content = section.text() # Navigate subsections for subsection in section.children: # Process hierarchically pass` API Reference ------------- ### DocumentSearch `DocumentSearch(document, use_cache=True)` Creates a search interface for a document. **Parameters:** - `document`: Parsed SEC document - `use_cache` (bool): Enable index caching (default: True) ### ranked\_search() `searcher.ranked_search( query: str, algorithm: str = "hybrid", top_k: int = 10, node_types: Optional[List[NodeType]] = None, in_section: Optional[str] = None, boost_sections: Optional[List[str]] = None ) -> List[SearchResult]` Perform ranked search with BM25-based ranking. **Parameters:** - `query`: Search query string - `algorithm`: Ranking algorithm ("bm25", "hybrid", "semantic") - `top_k`: Maximum results to return (default: 10) - `node_types`: Limit to specific node types (optional) - `in_section`: Limit to specific section (optional) - `boost_sections`: Sections to boost in ranking (optional) **Returns:** - List of `SearchResult` objects with scores and context ### get\_cache\_stats() `searcher.get_cache_stats() -> Dict[str, Any]` Get cache performance statistics. **Returns:** - Dictionary with cache metrics: - `memory_entries`: Indices in memory - `disk_entries`: Indices on disk - `cache_hits`: Total cache hits - `cache_misses`: Total cache misses - `hit_rate`: Cache hit rate (0-1) - `memory_size_mb`: Memory usage in MB ### clear\_cache() `searcher.clear_cache(memory_only: bool = False)` Clear search caches. **Parameters:** - `memory_only`: If True, only clear memory cache (default: False) Examples -------- See [ranking\_search\_examples.py](https://edgartools.readthedocs.io/en/stable/examples/ranking_search_examples.py) for comprehensive examples including: 1. Basic BM25 ranked search 2. Hybrid search with structure boosting 3. Semantic structure search 4. Section-specific search 5. Section boosting 6. Cache performance demonstration 7. Agent-friendly workflows 8. Comparing algorithms 9. Disabling cache 10. Cache management Migration from Old Search ------------------------- If you're currently using the basic `search()` method: ### Old Way (Basic Text Search) `results = searcher.search( query="revenue", mode=SearchMode.TEXT, limit=10 )` ### New Way (Ranked Search) `results = searcher.ranked_search( query="revenue growth trends", algorithm="hybrid", top_k=10 )` **Benefits:** - Relevance scores (not just presence/absence) - Structure-aware boosting - Better results for financial documents - 10x faster with caching - Full section context **Note:** The old `search()` method is still available for backwards compatibility. Troubleshooting --------------- ### Cache Not Working Check if caching is enabled: `searcher = DocumentSearch(document, use_cache=True) # Make sure use_cache=True` ### Memory Issues Reduce cache size: `from edgar.documents.ranking.cache import SearchIndexCache, set_search_cache cache = SearchIndexCache(memory_cache_size=5) # Reduce from default 10 set_search_cache(cache)` Or disable disk cache: `cache = SearchIndexCache(disk_cache_enabled=False) set_search_cache(cache)` ### Slow First Search First search builds the index (0.2-1.0s depending on document size). Subsequent searches are instant (~0.05s). This is normal and expected - the index is cached for future searches. Technical Details ----------------- ### BM25 Algorithm EdgarTools uses the Okapi BM25 variant with default parameters: - k1 = 1.5 (term frequency saturation) - b = 0.75 (length normalization) These parameters are optimized for financial documents. ### Caching Strategy * **Memory cache**: LRU eviction, configurable size (default: 10) * **Disk cache**: Pickle serialization in `~/.edgar_cache/search/` * **TTL**: 24 hours default (configurable) * **Index data**: Tokenized corpus + parameters (~5MB per index) ### Semantic Boosting Structure-aware boosting uses: - Node type scoring (headings > text > etc.) - Semantic type detection (item headers, section headers) - Cross-reference detection (regex patterns for "See Item X") - Position importance (earlier sections ranked higher) See Also -------- * [Document Parsing](https://edgartools.readthedocs.io/en/stable/advanced-search/parsing-filing-data.md) * [XBRL Querying](https://edgartools.readthedocs.io/en/stable/xbrl-querying/) * [Examples](https://edgartools.readthedocs.io/en/stable/examples/ranking_search_examples.py) Back to top --- # Advanced Search - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/advanced-search/#advanced-ranking-search) Advanced Ranking Search ======================= EdgarTools provides advanced search capabilities with BM25-based ranking and semantic structure awareness. This is designed specifically for financial documents to help you find the most relevant information quickly. Overview -------- The ranking search system offers: * **Three ranking algorithms**: BM25 (text-focused), Hybrid (text + structure), and Semantic (structure-focused) * **Intelligent caching**: 10x+ faster repeated searches with automatic index caching * **Structure-aware boosting**: Prioritizes headings, cross-references, and gateway content * **Agent-friendly results**: Full section context for investigation and navigation * **Performance optimized**: Instant results from cache, minimal memory overhead Quick Start ----------- `from edgar import Company from edgar.documents.search import DocumentSearch # Get a filing company = Company("AAPL") filing = company.get_filings(form="10-K").latest(1) # Parse HTML into structured Document document = filing.parse() # Create search interface searcher = DocumentSearch(document) # Search with ranking results = searcher.ranked_search( query="revenue growth", algorithm="hybrid", top_k=5 ) # Access results for result in results: print(f"Score: {result.score:.3f}") print(f"Section: {result.section}") print(f"Snippet: {result.snippet}")` About `filing.parse()` The `parse()` method parses the filing's HTML into a structured `Document` object with a node tree that `DocumentSearch` can index. This is different from `filing.document` which returns an `Attachment` (file metadata). Other useful Filing methods: * `filing.html()` - Returns raw HTML string * `filing.text()` - Returns plain text extraction * `filing.xbrl()` - Returns parsed XBRL data * `filing.parse()` - Returns structured Document for advanced operations Ranking Algorithms ------------------ ### BM25 (Best for Exact Term Matching) BM25 is a probabilistic retrieval function that ranks documents based on term frequency and inverse document frequency. It's excellent for finding exact matches of financial terms and concepts. `results = searcher.ranked_search( query="operating expenses depreciation", algorithm="bm25", top_k=10 )` **Best for:** - Finding specific financial terms - Exact phrase matching - Traditional keyword search **Parameters:** - `k1` (default: 1.5): Controls term frequency saturation - `b` (default: 0.75): Controls document length normalization ### Hybrid (Recommended for Most Use Cases) Hybrid combines BM25 text matching with semantic structure boosting. It understands document structure and boosts: * **Headings and section markers** (e.g., "Item 1A - Risk Factors") * **Cross-references** (e.g., "See Item 7 for discussion") * **Gateway content** (summaries, overviews, introductions) * **Tables and XBRL data** (structured financial information) `results = searcher.ranked_search( query="cybersecurity risks", algorithm="hybrid", top_k=5 )` **Best for:** - General-purpose search - Finding gateway content for investigation - Balancing exact matches with structural importance - Agent/LLM workflows **Weights (customizable):** - `bm25_weight` (default: 0.8): Weight for text matching - `semantic_weight` (default: 0.2): Weight for structure boosting ### Semantic (Best for Structure Navigation) Semantic ranking prioritizes document structure without text matching. It finds structurally important sections regardless of query terms. `results = searcher.ranked_search( query="business overview", algorithm="semantic", top_k=5 )` **Best for:** - Understanding document organization - Finding section boundaries - Structural navigation - Overview and summary content Advanced Search Options ----------------------- ### Section-Specific Search Limit search to specific sections: `results = searcher.ranked_search( query="supply chain risks", in_section="Risk Factors", top_k=5 )` ### Section Boosting Give higher weight to matches in certain sections: `results = searcher.ranked_search( query="revenue recognition", algorithm="hybrid", boost_sections=["MD&A", "Critical Accounting Policies"], top_k=5 )` ### Node Type Filtering Search only specific node types: `from edgar.documents.types import NodeType results = searcher.ranked_search( query="financial data", node_types=[NodeType.TABLE, NodeType.XBRL], top_k=5 )` Working with Results -------------------- Each result contains: `result.score # Relevance score (higher = more relevant) result.snippet # Short text snippet (first 200 chars) result.section # Section name (e.g., "Risk Factors") result.node # Original document node result.context # Full text context (up to 500 chars)` ### Accessing Full Context For agent workflows, results include full section access: `results = searcher.ranked_search("AI strategy", algorithm="hybrid") for result in results: # Access full section for investigation if hasattr(result, '_section_obj') and result._section_obj: section = result._section_obj full_text = section.text() # Navigate section structure for child in section.children: # Process subsections pass` Performance and Caching ----------------------- ### How Caching Works EdgarTools automatically caches search indices for fast repeated searches: 1. **Instance cache**: Stores engines for same DocumentSearch session 2. **Global cache**: Stores indices across documents (memory + disk) 3. **LRU eviction**: Automatically manages memory (default: 10 cached indices) 4. **TTL expiration**: Automatic cleanup after 24 hours ### Cache Performance Typical speedup: `import time # First search (cold cache) - builds index start = time.perf_counter() results1 = searcher.ranked_search("revenue", algorithm="bm25") cold_time = time.perf_counter() - start # ~0.5s # Second search (warm cache) - uses cached index start = time.perf_counter() results2 = searcher.ranked_search("revenue", algorithm="bm25") warm_time = time.perf_counter() - start # ~0.05s # 10x faster!` ### Cache Statistics Monitor cache performance: `stats = searcher.get_cache_stats() print(f"Cache hits: {stats['global_cache_stats']['cache_hits']}") print(f"Cache misses: {stats['global_cache_stats']['cache_misses']}") print(f"Hit rate: {stats['global_cache_stats']['hit_rate']:.1%}") print(f"Memory usage: {stats['global_cache_stats']['memory_size_mb']:.2f} MB")` ### Cache Management `# Clear instance cache only searcher.clear_cache(memory_only=True) # Clear all caches (memory + disk) searcher.clear_cache(memory_only=False) # Disable caching (for testing) searcher = DocumentSearch(document, use_cache=False)` ### Custom Cache Configuration `from edgar.documents.ranking.cache import SearchIndexCache, set_search_cache # Create custom cache cache = SearchIndexCache( memory_cache_size=20, # Store 20 indices in memory disk_cache_enabled=True, # Enable disk persistence ttl_hours=48 # Keep cached for 48 hours ) # Set as global cache set_search_cache(cache)` Best Practices -------------- ### Choosing the Right Algorithm | Use Case | Algorithm | Why | | --- | --- | --- | | Finding specific terms | BM25 | Exact text matching | | General document search | Hybrid | Balance text + structure | | Understanding document structure | Semantic | Pure structure focus | | Agent/LLM workflows | Hybrid | Finds gateway content | | Financial term lookup | BM25 | Best for exact matches | ### Performance Tips 1. **Use caching** (enabled by default) for repeated searches 2. **Use Hybrid algorithm** for most use cases (best results) 3. **Filter by section** to reduce search space 4. **Limit top\_k** to needed results (default: 10) 5. **Monitor cache stats** to optimize cache size ### Agent Workflows For AI agents investigating documents: `# Step 1: Find relevant sections results = searcher.ranked_search( query="climate risk disclosures", algorithm="hybrid", top_k=3 ) # Step 2: Investigate full sections for result in results: if result._section_obj: section = result._section_obj # Read full section full_content = section.text() # Navigate subsections for subsection in section.children: # Process hierarchically pass` API Reference ------------- ### DocumentSearch `DocumentSearch(document, use_cache=True)` Creates a search interface for a document. **Parameters:** - `document`: Parsed SEC document - `use_cache` (bool): Enable index caching (default: True) ### ranked\_search() `searcher.ranked_search( query: str, algorithm: str = "hybrid", top_k: int = 10, node_types: Optional[List[NodeType]] = None, in_section: Optional[str] = None, boost_sections: Optional[List[str]] = None ) -> List[SearchResult]` Perform ranked search with BM25-based ranking. **Parameters:** - `query`: Search query string - `algorithm`: Ranking algorithm ("bm25", "hybrid", "semantic") - `top_k`: Maximum results to return (default: 10) - `node_types`: Limit to specific node types (optional) - `in_section`: Limit to specific section (optional) - `boost_sections`: Sections to boost in ranking (optional) **Returns:** - List of `SearchResult` objects with scores and context ### get\_cache\_stats() `searcher.get_cache_stats() -> Dict[str, Any]` Get cache performance statistics. **Returns:** - Dictionary with cache metrics: - `memory_entries`: Indices in memory - `disk_entries`: Indices on disk - `cache_hits`: Total cache hits - `cache_misses`: Total cache misses - `hit_rate`: Cache hit rate (0-1) - `memory_size_mb`: Memory usage in MB ### clear\_cache() `searcher.clear_cache(memory_only: bool = False)` Clear search caches. **Parameters:** - `memory_only`: If True, only clear memory cache (default: False) Examples -------- See [ranking\_search\_examples.py](https://edgartools.readthedocs.io/en/latest/examples/ranking_search_examples.py) for comprehensive examples including: 1. Basic BM25 ranked search 2. Hybrid search with structure boosting 3. Semantic structure search 4. Section-specific search 5. Section boosting 6. Cache performance demonstration 7. Agent-friendly workflows 8. Comparing algorithms 9. Disabling cache 10. Cache management Migration from Old Search ------------------------- If you're currently using the basic `search()` method: ### Old Way (Basic Text Search) `results = searcher.search( query="revenue", mode=SearchMode.TEXT, limit=10 )` ### New Way (Ranked Search) `results = searcher.ranked_search( query="revenue growth trends", algorithm="hybrid", top_k=10 )` **Benefits:** - Relevance scores (not just presence/absence) - Structure-aware boosting - Better results for financial documents - 10x faster with caching - Full section context **Note:** The old `search()` method is still available for backwards compatibility. Troubleshooting --------------- ### Cache Not Working Check if caching is enabled: `searcher = DocumentSearch(document, use_cache=True) # Make sure use_cache=True` ### Memory Issues Reduce cache size: `from edgar.documents.ranking.cache import SearchIndexCache, set_search_cache cache = SearchIndexCache(memory_cache_size=5) # Reduce from default 10 set_search_cache(cache)` Or disable disk cache: `cache = SearchIndexCache(disk_cache_enabled=False) set_search_cache(cache)` ### Slow First Search First search builds the index (0.2-1.0s depending on document size). Subsequent searches are instant (~0.05s). This is normal and expected - the index is cached for future searches. Technical Details ----------------- ### BM25 Algorithm EdgarTools uses the Okapi BM25 variant with default parameters: - k1 = 1.5 (term frequency saturation) - b = 0.75 (length normalization) These parameters are optimized for financial documents. ### Caching Strategy * **Memory cache**: LRU eviction, configurable size (default: 10) * **Disk cache**: Pickle serialization in `~/.edgar_cache/search/` * **TTL**: 24 hours default (configurable) * **Index data**: Tokenized corpus + parameters (~5MB per index) ### Semantic Boosting Structure-aware boosting uses: - Node type scoring (headings > text > etc.) - Semantic type detection (item headers, section headers) - Cross-reference detection (regex patterns for "See Item X") - Position importance (earlier sections ranked higher) See Also -------- * [Document Parsing](https://edgartools.readthedocs.io/en/latest/advanced-search/parsing-filing-data.md) * [XBRL Querying](https://edgartools.readthedocs.io/en/latest/xbrl-querying/) * [Examples](https://edgartools.readthedocs.io/en/latest/examples/ranking_search_examples.py) Back to top --- # Customizing Standardization - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/advanced/customizing-standardization/#customizing-xbrl-standardization) Customizing XBRL Standardization ================================ **Target Audience**: Advanced users, financial analysts, quantitative researchers **Prerequisites**: Understanding of XBRL concepts, Python basics, JSON format **Use Case**: Custom taxonomies, 200+ companies, industry-specific valuations * * * Table of Contents ----------------- 1. [Overview and Introduction](https://edgartools.readthedocs.io/en/latest/advanced/customizing-standardization/#1-overview-and-introduction) 2. [Architecture and Design](https://edgartools.readthedocs.io/en/latest/advanced/customizing-standardization/#2-architecture-and-design) 3. [Core Mappings Structure](https://edgartools.readthedocs.io/en/latest/advanced/customizing-standardization/#3-core-mappings-structure) 4. [Company-Specific Mappings](https://edgartools.readthedocs.io/en/latest/advanced/customizing-standardization/#4-company-specific-mappings) 5. [Priority System and Ambiguous Tag Resolution](https://edgartools.readthedocs.io/en/latest/advanced/customizing-standardization/#5-priority-system-and-ambiguous-tag-resolution) 6. [Current Limitations](https://edgartools.readthedocs.io/en/latest/advanced/customizing-standardization/#6-current-limitations) 7. [Validation Techniques](https://edgartools.readthedocs.io/en/latest/advanced/customizing-standardization/#7-validation-techniques) 8. [CSV Workflow](https://edgartools.readthedocs.io/en/latest/advanced/customizing-standardization/#8-csv-workflow) 9. [Real-World Examples](https://edgartools.readthedocs.io/en/latest/advanced/customizing-standardization/#9-real-world-examples) 10. [Future Enhancements](https://edgartools.readthedocs.io/en/latest/advanced/customizing-standardization/#10-future-enhancements) * * * 1\. Overview and Introduction ----------------------------- ### What is XBRL Standardization? XBRL standardization is the process of mapping company-specific XBRL tags to a consistent set of standardized concept names. This enables: * **Consistent presentation** of financial statements across different companies * **Comparable analysis** regardless of each company's unique taxonomy * **Automated processing** of financial data from diverse sources * **Reduced complexity** when working with 200+ companies ### Why Companies Need Custom Taxonomies Every company's XBRL filing uses a mix of: - **US-GAAP standard tags**: `us-gaap:Revenue`, `us-gaap:Assets` - **Company-specific extensions**: `tsla:AutomotiveRevenue`, `msft:AzureRevenue` - **Industry-specific concepts**: Energy, automotive, technology sectors **The Problem**: Without standardization, analyzing 200 companies means dealing with thousands of unique XBRL tag variations for the same underlying financial concept. **The Solution**: EdgarTools' standardization system maps all these variations to a unified set of standard concepts. ### When to Customize Standardization You should customize the standardization system when: * **Managing 200+ companies** with diverse taxonomies * **Working with industry-specific filings** (automotive, technology, industrial firms) * **Building valuation models** requiring granular financial statements * **Conducting multi-company analysis** that requires consistent data structure * **Ensuring statement balancing** (Assets = Liabilities + Equity) across diverse filings ### What This Guide Covers This comprehensive guide explains: - How the standardization architecture works - How to create custom concept mappings - How to handle ambiguous XBRL tags (200+ identified cases) - How to validate mapping quality - Production-ready workflows for managing custom taxonomies * * * 2\. Architecture and Design --------------------------- ### The StandardConcept Enum vs JSON Mappings This is a critical distinction that causes confusion: #### StandardConcept Enum: IDE Convenience (Optional) `from edgar.xbrl.standardization import StandardConcept # Enum provides autocomplete and type safety revenue_label = StandardConcept.REVENUE.value # "Revenue" assets_label = StandardConcept.TOTAL_ASSETS.value # "Total Assets"` **Purpose**: - IDE autocomplete for known concepts - Type safety for Python code - Semantic meaning for core financial concepts **Location**: `edgar/xbrl/standardization/core.py` (lines 18-126) #### JSON Mappings: Source of Truth (Required) `{ "Revenue": [ "us-gaap:Revenue", "us-gaap:Revenues", "us-gaap:SalesRevenueNet" ] }` **Purpose**: - The actual mapping data used by the system - Unlimited extensibility without code changes - User-customizable without touching Python code **Location**: `edgar/xbrl/standardization/concept_mappings.json` #### Critical Clarification **The Relationship**: - Enum values SHOULD match JSON keys (e.g., `StandardConcept.REVENUE.value == "Revenue"`) - This relationship is NOT enforced by code - JSON is what the system actually uses for mapping - Enum is purely for developer convenience **For Custom Mappings**: - **You customize via JSON files** - NOT by modifying the enum - The enum can remain unchanged; JSON drives all behavior - You can add mappings in JSON that don't exist in the enum - System will validate JSON keys against enum if you enable validation (optional) ### How Standardization Works The standardization system follows this flow: `Company XBRL Tag → MappingStore → Priority Resolution → Standard Concept` **Example**: `"tsla:AutomotiveRevenue" → [Priority 4: Tesla mapping] → "Automotive Revenue" "us-gaap:Revenue" → [Priority 1: Core mapping] → "Revenue"` ### Key Components #### 1\. MappingStore (The Brain) **File**: `edgar/xbrl/standardization/core.py` (lines 128-462) **Responsibilities**: - Loads core mappings from `concept_mappings.json` - Loads company-specific mappings from `company_mappings/` directory - Merges mappings with priority scoring - Resolves ambiguous tags using context **Initialization**: `from edgar.xbrl.standardization import MappingStore # Default initialization (loads packaged mappings) store = MappingStore() # Custom source (future enhancement) store = MappingStore(source="/path/to/custom_mappings.json") # Read-only mode (for testing) store = MappingStore(read_only=True)` #### 2\. ConceptMapper (The Worker) **File**: `edgar/xbrl/standardization/core.py` (lines 464-682) **Responsibilities**: - Maps individual concepts using MappingStore - Caches results for performance - Handles context-aware inference - Tracks unmapped concepts **Usage**: `mapper = ConceptMapper(mapping_store) # Map a concept with context context = { 'statement_type': 'BalanceSheet', 'level': 0, 'is_total': True } standard_concept = mapper.map_concept( company_concept='us-gaap:Assets', label='Total Assets', context=context ) # Returns: "Total Assets"` #### 3\. Priority System The system uses priority levels to resolve conflicts: | Priority | Source | Description | Example | | --- | --- | --- | --- | | **P1** | Core mappings | Base US-GAAP concepts | `us-gaap:Revenue → "Revenue"` | | **P2** | Company mappings | Company-specific overrides | `tsla:Revenue → "Automotive Revenue"` | | **P4** | Detected entity | Auto-detected from prefix | `tsla:CustomTag → uses Tesla P2 mappings` | **Priority Resolution Algorithm** (lines 408-449): 1. Detect entity from concept prefix (e.g., `tsla:` → `"tsla"`) 2. Search through merged mappings 3. For each match, calculate effective priority 4. If detected entity matches mapping source, boost to P4 5. Return highest priority match * * * 3\. Core Mappings Structure --------------------------- ### File Location **Current Location** (hardcoded): `edgar/xbrl/standardization/concept_mappings.json` **Future Enhancement** (v4.30.0): Configurable paths via environment variables. ### JSON Structure The core mappings file uses a flat dictionary structure: `{ "Standard Concept Label": [ "company_specific_tag_1", "company_specific_tag_2", "us-gaap:StandardTag" ], "_comment_section": "Documentation comments for maintainers" }` ### Real Example from concept\_mappings.json `{ "_comment_revenue_hierarchy": "REVENUE HIERARCHY FIX: Separated total revenue from component revenue types to prevent duplicate labels.", "Revenue": [ "us-gaap:Revenue", "us-gaap:Revenues", "us-gaap:SalesRevenueNet", "us-gaap:OperatingRevenue" ], "Contract Revenue": [ "us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax", "us-gaap:RevenueFromContractWithCustomerIncludingAssessedTax" ], "Product Revenue": [ "us-gaap:SalesRevenueGoodsNet", "us-gaap:ProductSales" ] }` ### Understanding Comments The JSON file includes `_comment_*` keys for documentation: - These are ignored by the mapping system - They explain design decisions - They help maintainers understand complex hierarchies ### Hierarchy Separation Notice the careful separation of concepts: - **"Revenue"**: Total revenue (parent concept) - **"Contract Revenue"**: Component of revenue (child concept) - **"Product Revenue"**: Another component (sibling to Contract Revenue) This prevents mapping conflicts where multiple XBRL tags map to the same label. ### Cost Hierarchy Example `{ "_comment_cost_of_revenue_hierarchy": "Different business models use different cost concepts that should have distinct labels.", "Total Cost of Revenue": [ "us-gaap:CostOfRevenue" ], "Cost of Goods Sold": [ "us-gaap:CostOfGoodsSold" ], "Cost of Goods and Services Sold": [ "us-gaap:CostOfGoodsAndServicesSold" ], "Direct Operating Costs": [ "us-gaap:DirectOperatingCosts" ] }` **Why separate these?** - Manufacturing companies use "Cost of Goods Sold" - Service companies use "Direct Operating Costs" - Mixed businesses use "Cost of Goods and Services Sold" - Each should have a distinct label for clarity ### Adding Custom Core Mappings To extend core mappings (not recommended for most users): 1. **Locate the file**: `edgar/xbrl/standardization/concept_mappings.json` 2. **Add your mapping**: `{ "Custom Concept Label": [ "us-gaap:YourCustomTag", "company:AnotherTag" ] }` 3. **Maintain hierarchy**: Ensure parent/child relationships are clear 4. **Add comments**: Document your reasoning with `_comment_*` keys **Warning**: Modifying packaged files is not recommended. Use company-specific mappings instead (Section 4). * * * 4\. Company-Specific Mappings ----------------------------- ### Why Company-Specific Mappings? Company-specific mapping files allow you to: - Override core mappings for specific companies - Add industry-specific concepts (automotive, technology, energy) - Handle company extension taxonomies - Maintain separation of concerns (one file per company) ### File Structure: {ticker}\_mappings.json **Current Location** (hardcoded): `edgar/xbrl/standardization/company_mappings/{ticker}_mappings.json` **Important Note**: Currently uses ticker as identifier, but **CIK-based identification is coming in v4.30.0/v4.31.0** to handle multi-ticker companies (GOOG/GOOGL, HEI.A/HEI.B). ### Complete Company Mapping Schema `{ "metadata": { "entity_identifier": "ticker_symbol", "company_name": "Full Company Name", "cik": "1234567", "priority": "high|medium|low", "created_date": "YYYY-MM-DD", "last_updated": "YYYY-MM-DD", "description": "Brief description of custom taxonomy needs" }, "concept_mappings": { "Standard Concept Label": [ "company:CustomTag", "company:AnotherCustomTag" ] }, "hierarchy_rules": { "Parent Concept": { "children": [ "Child Concept 1", "Child Concept 2" ], "description": "Optional explanation" } }, "business_context": { "primary_revenue_streams": ["stream1", "stream2"], "revenue_model": "product_and_service|subscription|manufacturing", "key_metrics": ["metric1", "metric2"], "industry": "industry_classification" } }` ### Real Example: Tesla (tsla\_mappings.json) `{ "metadata": { "entity_identifier": "tsla", "company_name": "Tesla, Inc.", "cik": "1318605", "priority": "high", "created_date": "2024-06-25", "last_updated": "2024-06-25", "description": "Tesla-specific concept mappings to handle automotive, energy, and service revenue streams" }, "concept_mappings": { "Automotive Revenue": [ "tsla:AutomotiveRevenue", "tsla:AutomotiveSales", "tsla:VehicleRevenue" ], "Automotive Leasing Revenue": [ "tsla:AutomotiveLeasing", "tsla:AutomotiveLeasingRevenue", "tsla:VehicleLeasingRevenue" ], "Energy Revenue": [ "tsla:EnergyGenerationAndStorageRevenue", "tsla:EnergyRevenue", "tsla:SolarRevenue", "tsla:EnergyStorageRevenue" ], "Service Revenue": [ "tsla:ServicesAndOtherRevenue", "tsla:ServiceRevenue", "tsla:SuperchargerRevenue" ] }, "hierarchy_rules": { "Revenue": { "children": [ "Automotive Revenue", "Energy Revenue", "Service Revenue" ] }, "Automotive Revenue": { "children": [ "Automotive Leasing Revenue" ] } }, "business_context": { "primary_revenue_streams": ["automotive", "energy", "services"], "revenue_model": "product_and_service", "key_metrics": ["vehicle_deliveries", "energy_deployments"], "industry": "automotive_technology" } }` ### Real Example: Microsoft (msft\_mappings.json) `{ "entity_info": { "name": "Microsoft Corporation", "cik": "0000789019", "ticker": "MSFT", "description": "Microsoft-specific concept mappings for unique business terminology" }, "concept_mappings": { "_comment_msft_revenue": "Microsoft uses specific revenue categorization that differs from standard tech companies", "Product Revenue": [ "msft:ProductRevenue", "msft:WindowsCommercialRevenue", "msft:WindowsConsumerRevenue", "msft:OfficeCommercialRevenue" ], "Service Revenue": [ "msft:ServiceRevenue", "msft:CloudServicesRevenue", "msft:ConsultingServicesRevenue" ], "Subscription Revenue": [ "msft:Office365CommercialRevenue", "msft:Office365ConsumerRevenue", "msft:DynamicsRevenue" ], "Platform Revenue": [ "msft:AzureRevenue", "msft:XboxContentAndServicesRevenue" ], "_comment_msft_expenses": "Microsoft has unique expense categorizations", "Sales and Marketing Expense": [ "msft:SalesAndMarketingExpense", "msft:AdvertisingAndPromotionExpense" ], "Technical Support Expense": [ "msft:TechnicalSupportExpense", "msft:CustomerSupportExpense" ] }, "hierarchy_rules": { "_comment": "Rules for handling Microsoft-specific hierarchical relationships", "revenue_hierarchy": { "parent": "Revenue", "children": ["Product Revenue", "Service Revenue", "Subscription Revenue", "Platform Revenue"], "calculation_rule": "sum" }, "expense_hierarchy": { "parent": "Operating Expenses", "children": ["Sales and Marketing Expense", "Technical Support Expense"], "calculation_rule": "sum" } } }` ### Creating Your Own Company Mapping **Step 1: Identify Company Extension Tags** `from edgar import Company company = Company("AAPL") filing = company.get_filings(form="10-K").latest() xbrl = filing.xbrl() # Find company-specific tags facts = xbrl.facts.query().to_dataframe() company_tags = facts[facts['concept'].str.startswith('aapl:')]['concept'].unique() print(f"Found {len(company_tags)} Apple-specific tags")` **Step 2: Create Mapping File** `{ "metadata": { "entity_identifier": "aapl", "company_name": "Apple Inc.", "cik": "0000320193", "priority": "high", "created_date": "2025-11-19", "last_updated": "2025-11-19", "description": "Apple-specific mappings for product categories" }, "concept_mappings": { "iPhone Revenue": [ "aapl:IPhoneRevenue", "aapl:IPhoneSales" ], "Services Revenue": [ "aapl:ServicesRevenue", "aapl:AppleCareRevenue", "aapl:ICloudRevenue" ] } }` **Step 3: Place File in Correct Location** `edgar/xbrl/standardization/company_mappings/aapl_mappings.json` **Current Limitation**: Must be inside the package directory (see Section 6). * * * 5\. Priority System and Ambiguous Tag Resolution ------------------------------------------------ ### The Ambiguous Tag Problem Over **200 XBRL tags** are inherently ambiguous and can map to multiple standard concepts depending on context. These fall into several categories: #### Category 1: Asset/Liability Ambiguity (12 tags) Tags that could be either assets or liabilities: `DeferredTaxAssetsLiabilitiesNet DeferredTaxAssetsLiabilitiesNetCurrent DeferredTaxAssetsLiabilitiesNetNoncurrent DerivativeAssetsLiabilitiesAtFairValueNet DeferredFinanceCostsCurrentNet DeferredFinanceCostsNoncurrentNet CustomerAdvancesAndProgressPaymentsForLongTermContractsOrPrograms DeferredTaxLiabilitiesGoodwillAndIntangibleAssets DeferredTaxLiabilitiesInvestments UnamortizedDebtIssuanceExpense DerivativeLiabilityFairValueGrossAsset` **Example: DeferredTaxAssetsLiabilitiesNet** This tag represents the NET of deferred tax assets and liabilities: - If positive → Deferred Tax Asset - If negative → Deferred Tax Liability **Resolution Strategy**: Use statement context and sign convention. #### Category 2: Current/Noncurrent Ambiguity (180+ tags) Tags that don't specify classification: `AccountsPayableCurrentAndNoncurrent AccountsPayableAndAccruedLiabilitiesCurrentAndNoncurrent AccountsReceivableGross AccountsReceivableNet DeferredRevenue ContractWithCustomerLiability ConvertibleDebt DebtInstrumentCarryingAmount` **Example: AccountsPayableCurrentAndNoncurrent** Without context, you can't determine if this should map to: - "Accounts Payable, Current" (on current liabilities) - "Accounts Payable, Noncurrent" (on long-term liabilities) - "Accounts Payable, Total" (parent concept) **Resolution Strategy**: Use calculation tree parent relationships and statement location. #### Category 3: Triple Ambiguity (1 tag) `DerivativeLiabilityFairValueGrossAsset` This tag is ambiguous in THREE dimensions: 1. Asset vs. Liability 2. Current vs. Noncurrent 3. Gross vs. Net **Resolution**: Requires comprehensive context analysis. #### Category 4: Total vs. Line Item Ambiguity `LiabilitiesNoncurrent` Some companies use this as: - **Total line**: Sum of all noncurrent liabilities - **Other line item**: "Other Noncurrent Liabilities" **Resolution**: Check if it has children in calculation tree or appears multiple times. ### Priority Levels Explained #### Priority 1: Core Mappings **Source**: `concept_mappings.json` **Use**: Base US-GAAP concepts that apply to all companies `# P1 mapping example "Revenue": [ "us-gaap:Revenue", "us-gaap:Revenues", "us-gaap:SalesRevenueNet" ]` **When Applied**: When no company-specific mapping exists. #### Priority 2: Company Mappings **Source**: `company_mappings/{ticker}_mappings.json` **Use**: Company-specific overrides and extensions `# P2 mapping example (Tesla) "Automotive Revenue": [ "tsla:AutomotiveRevenue", "tsla:AutomotiveSales" ]` **When Applied**: When company mapping file exists for the ticker. #### Priority 4: Detected Entity Match **Source**: Automatic detection from concept prefix **Use**: Boost priority when concept prefix matches company `# P4 boost example concept = "tsla:AutomotiveRevenue" # System detects "tsla:" prefix # Checks if "tsla" company mappings exist # Boosts priority from P2 → P4 for Tesla mappings` **When Applied**: When concept prefix matches a known company identifier. ### Context-Based Resolution The system uses multiple context signals to resolve ambiguous tags: #### 1\. Statement Type Context `context = { 'statement_type': 'BalanceSheet' # or 'IncomeStatement', 'CashFlowStatement' }` **Example Resolution**: `Tag: "DeferredTaxAssetsLiabilitiesNet" Statement: BalanceSheet Parent: "Assets" → Maps to: "Deferred Tax Assets" Tag: "DeferredTaxAssetsLiabilitiesNet" Statement: BalanceSheet Parent: "Liabilities" → Maps to: "Deferred Tax Liabilities"` #### 2\. Calculation Tree Relationships `context = { 'calculation_parent': 'us-gaap:AssetsCurrent', 'level': 1 }` **Example Resolution**: `Tag: "AccountsPayableCurrentAndNoncurrent" Parent: "AssetsCurrent" (impossible - payables are liabilities) → Check if sign is negative → Maps to: "Accounts Payable, Current" (negative in assets = liability) Tag: "AccountsPayableCurrentAndNoncurrent" Parent: "LiabilitiesCurrent" → Maps to: "Accounts Payable, Current" Tag: "AccountsPayableCurrentAndNoncurrent" Parent: "LiabilitiesNoncurrent" → Maps to: "Accounts Payable, Noncurrent"` #### 3\. Sign Conventions `# Positive value in assets section → Asset # Negative value in assets section → Liability (unusual presentation) # Check fact value and location` #### 4\. Position and Level `context = { 'position': 0, # First item in section 'level': 0, # Top level (total) 'is_total': True }` **Example Resolution**: `Tag: "LiabilitiesNoncurrent" Level: 0 Has children: Yes → Maps to: "Total Noncurrent Liabilities" Tag: "LiabilitiesNoncurrent" Level: 1 Has children: No → Maps to: "Other Noncurrent Liabilities"` ### Complete Ambiguous Tag List For reference, here are all identified ambiguous tags (from user @mpreiss9's analysis): **Asset/Liability Ambiguity (12 tags)**: `CustomerAdvancesAndProgressPaymentsForLongTermContractsOrPrograms DeferredFinanceCostsCurrentNet DeferredFinanceCostsNoncurrentNet DeferredTaxAssetsLiabilitiesNet DeferredTaxAssetsLiabilitiesNetCurrent DeferredTaxAssetsLiabilitiesNetNoncurrent DeferredTaxLiabilitiesGoodwillAndIntangibleAssets DeferredTaxLiabilitiesGoodwillAndIntangibleAssetsIntangibleAssets DeferredTaxLiabilitiesInvestments DerivativeAssetsLiabilitiesAtFairValueNet UnamortizedDebtIssuanceExpense DerivativeLiabilityFairValueGrossAsset` **Current/Noncurrent Ambiguity (180+ tags)** - Excerpt: `AccountsPayableAndAccruedLiabilitiesCurrentAndNoncurrent AccountsPayableAndOtherAccruedLiabilities AccountsPayableCurrentAndNoncurrent AccountsPayableOtherCurrentAndNoncurrent AccountsPayableTradeCurrentAndNoncurrent AccountsReceivableGross AccountsReceivableNet AccountsReceivableRelatedParties AccrualForTaxesOtherThanIncomeTaxesCurrentAndNoncurrent AccruedAdvertisingCurrentAndNoncurrent AccruedBonusesCurrentAndNoncurrent AccruedEmployeeBenefitsCurrentAndNoncurrent AccruedIncomeTaxes AccruedLiabilitiesCurrentAndNoncurrent AvailableForSaleSecuritiesDebtSecurities BusinessCombinationContingentConsiderationAsset BusinessCombinationContingentConsiderationLiability CapitalizedContractCostNet CapitalLeaseObligations ContractWithCustomerAssetNet ContractWithCustomerLiability ConvertibleDebt DebtInstrumentCarryingAmount DeferredRevenue DeferredTaxAssetsGross DeferredTaxAssetsNet DeferredTaxLiabilities DeferredTaxLiabilitiesNet DerivativeAssets DerivativeLiabilities EquitySecuritiesFvNi HeldToMaturitySecurities Investments LineOfCredit MarketableSecurities NotesAndLoansPayable OperatingLeaseLiability OtherAssets OtherLiabilities RestrictedCash` **See issue #494 comment for complete list of 200+ tags**. ### Implementing Custom Ambiguity Resolution If you need to handle ambiguous tags for your specific use case: **Option 1: Company-Specific Mapping** `{ "concept_mappings": { "Deferred Tax Assets": [ "us-gaap:DeferredTaxAssetsLiabilitiesNet" ] }, "notes": { "DeferredTaxAssetsLiabilitiesNet": "Company X always reports net deferred tax assets; never reports net liability" } }` **Option 2: Context Validation (Future Enhancement)** This is planned for v4.30.0: `from edgar.xbrl.standardization import MappingStore store = MappingStore() # Custom resolution function def custom_resolver(concept, context, value): if concept == "us-gaap:DeferredTaxAssetsLiabilitiesNet": if value > 0: return "Deferred Tax Assets" else: return "Deferred Tax Liabilities" return None store.add_custom_resolver(custom_resolver)` * * * 6\. Current Limitations ----------------------- This section documents **known limitations** of the current implementation and provides workarounds. These are on the roadmap for future releases. ### Limitation 1: Hardcoded Paths **Problem**: Mapping files MUST be inside the package directory. **Current Paths**: `edgar/xbrl/standardization/concept_mappings.json edgar/xbrl/standardization/company_mappings/{ticker}_mappings.json` **Why This is a Problem**: - Users can't maintain mappings outside the package - Difficult to version control custom mappings separately - Package updates overwrite custom mappings - Not suitable for production deployment workflows **Current Workaround**: Copy your custom mapping files into the package directory after installation: `# Find package location python -c "import edgar; print(edgar.__file__)" # Output: /path/to/site-packages/edgar/__init__.py # Copy your custom mappings cp my_custom_mappings.json /path/to/site-packages/edgar/xbrl/standardization/ cp company_mappings/* /path/to/site-packages/edgar/xbrl/standardization/company_mappings/` **Better Workaround** (Python script): `import shutil from pathlib import Path import edgar # Find package standardization directory edgar_path = Path(edgar.__file__).parent std_path = edgar_path / "xbrl" / "standardization" # Copy custom core mappings shutil.copy( "my_custom_mappings.json", std_path / "concept_mappings.json" ) # Copy company mappings company_dir = std_path / "company_mappings" for mapping_file in Path("my_company_mappings").glob("*_mappings.json"): shutil.copy(mapping_file, company_dir / mapping_file.name) print("Custom mappings installed successfully")` **Risks**: - Mappings are lost on package upgrade - Must re-run after each `pip install --upgrade edgartools` **Future Enhancement** (v4.30.0): `# Coming in v4.30.0 import os os.environ['EDGAR_MAPPINGS_PATH'] = '/path/to/my/mappings' # Or via constructor from edgar.xbrl.standardization import MappingStore store = MappingStore( core_path='/path/to/concept_mappings.json', company_dir='/path/to/company_mappings/' )` ### Limitation 2: Ticker-Based Identification **Problem**: Company mappings use ticker as identifier, not CIK. **Current Behavior**: `# File: company_mappings/msft_mappings.json { "metadata": { "entity_identifier": "msft", # Uses ticker "cik": "0000789019" # CIK is just metadata } }` **Why This is a Problem**: 1. **Multiple tickers per CIK**: 2. Alphabet: GOOG and GOOGL → Same CIK (1652044) 3. HEICO: HEI.A and HEI.B → Same CIK (46619) 4. **Ticker changes over time**: 5. Facebook → Meta (FB → META) 6. Google → Alphabet (GOOG → GOOGL split) 7. **CIK is the stable identifier** in SEC filings: 8. Every filing contains CIK 9. Ticker can be ambiguous or absent **Current Workaround**: Use the **primary ticker** and document alternatives in metadata: `{ "metadata": { "entity_identifier": "goog", "company_name": "Alphabet Inc.", "cik": "0001652044", "alternative_tickers": ["GOOGL", "GOOG"], "notes": "Use GOOG as primary; applies to both Class A (GOOGL) and Class C (GOOG)" } }` Then create a symlink or duplicate file: `cd company_mappings/ cp goog_mappings.json googl_mappings.json` **Future Enhancement** (v4.30.0 - v4.31.0): `{ "metadata": { "entity_identifier": "0001652044", # CIK is primary identifier "tickers": ["GOOG", "GOOGL"], # Multiple tickers supported "primary_ticker": "GOOG" } }` **Timeline**: - **v4.30.0**: Add CIK-based lookup support (dual lookup during transition) - **v4.31.0**: Default to CIK-based identification - **v5.0.0**: Deprecate ticker-based identification ### Limitation 3: JSON-Only Format **Problem**: No native CSV support for mapping files. **Why This is a Problem**: - CSV is easier to edit in Excel or Google Sheets - Easier to detect duplicates with spreadsheet tools - Simpler to sort, filter, and validate mappings - More accessible for non-technical users **Current Workaround**: See Section 8 (CSV Workflow) for utilities. **Quick CSV-to-JSON Converter**: `import csv import json from collections import defaultdict def csv_to_mappings(csv_path, json_path): """Convert CSV mapping file to JSON format.""" mappings = defaultdict(list) with open(csv_path, 'r') as f: reader = csv.DictReader(f) for row in reader: standard_concept = row['standard_concept'] company_concept = row['company_concept'] mappings[standard_concept].append(company_concept) with open(json_path, 'w') as f: json.dump(dict(mappings), f, indent=2) print(f"Converted {len(mappings)} concepts") # Usage csv_to_mappings('my_mappings.csv', 'concept_mappings.json')` **Expected CSV Format**: `standard_concept,company_concept,notes Revenue,us-gaap:Revenue,Standard revenue tag Revenue,us-gaap:Revenues,Alternative spelling Automotive Revenue,tsla:AutomotiveRevenue,Tesla-specific` **Future Enhancement** (v4.30.0): `# Native CSV support - auto-detect from extension from edgar.xbrl.standardization import MappingStore # Automatically loads CSV or JSON based on extension store = MappingStore(source="my_mappings.csv")` * * * 7\. Validation Techniques ------------------------- Validation is critical when working with custom mappings across 200+ companies. Here are proven techniques for ensuring mapping quality. ### The Balance Sheet Validation Principle The fundamental validation for balance sheets: `# Core accounting equation Assets = Liabilities + Equity # Detailed validation Total Assets = Current Assets + Noncurrent Assets Total Assets = Sum(all individual asset line items) Total Liabilities = Current Liabilities + Noncurrent Liabilities Total Equity = Common Stock + Retained Earnings + Other Equity Assets = Liabilities + Equity` ### Balance Sheet Validation Code `def validate_balance_sheet(xbrl, period_key): """Validate that balance sheet balances using mapped concepts.""" facts = xbrl.facts.query().by_period_key(period_key).to_dataframe() # Get key totals total_assets = facts[facts['label'] == 'Total Assets']['value'].sum() current_assets = facts[facts['label'] == 'Total Current Assets']['value'].sum() noncurrent_assets = facts[facts['label'] == 'Total Non Current Assets']['value'].sum() total_liabilities = facts[facts['label'] == 'Total Liabilities']['value'].sum() total_equity = facts[facts['label'] == "Total Stockholders' Equity"]['value'].sum() # Validation 1: Assets = Current + Noncurrent if noncurrent_assets: # Some companies don't report noncurrent separately assets_check = abs(total_assets - (current_assets + noncurrent_assets)) < 1.0 if not assets_check: print(f"WARNING: Assets don't balance: {total_assets} != {current_assets} + {noncurrent_assets}") # Validation 2: Assets = Liabilities + Equity accounting_equation = abs(total_assets - (total_liabilities + total_equity)) < 1.0 if not accounting_equation: print(f"ERROR: Accounting equation violated: {total_assets} != {total_liabilities} + {total_equity}") return False # Validation 3: Sum of line items = Total Assets asset_line_items = facts[ (facts['concept'].str.contains('Asset')) & (facts['label'] != 'Total Assets') ]['value'].sum() detail_check = abs(total_assets - asset_line_items) < 1.0 if not detail_check: print(f"WARNING: Asset line items don't sum to total: {total_assets} != {asset_line_items}") return accounting_equation # Usage filing = Company("AAPL").get_filings(form="10-K").latest() xbrl = filing.xbrl() period_key = xbrl.reporting_periods[0]['key'] is_valid = validate_balance_sheet(xbrl, period_key) print(f"Balance sheet valid: {is_valid}")` ### Income Statement Validation Income statement validation is **more complex** due to: - Variable presentation formats - Sign convention inconsistencies (some expenses are positive, some negative) - Different levels of detail across companies `def validate_income_statement(xbrl, period_key): """Validate income statement using anchored approach.""" facts = xbrl.facts.query().by_period_key(period_key).to_dataframe() # Anchor points (always present and unambiguous) revenue = facts[facts['label'] == 'Revenue']['value'].sum() net_income = facts[facts['label'] == 'Net Income']['value'].sum() if revenue == 0 or net_income == 0: print("ERROR: Missing anchor points (Revenue or Net Income)") return False # Get all expense items (with sign normalization) expense_concepts = [ 'Cost of Revenue', 'Research and Development Expense', 'Selling, General and Administrative Expense', 'Interest Expense', 'Income Tax Expense' ] total_expenses = 0 for concept in expense_concepts: expense_facts = facts[facts['label'] == concept]['value'] if len(expense_facts) > 0: expense_value = expense_facts.sum() # Normalize to positive (expenses reduce income) if expense_value < 0: expense_value = abs(expense_value) total_expenses += expense_value # Check if Revenue - Expenses ≈ Net Income # Allow for other income/expense not captured calculated_ni = revenue - total_expenses difference = abs(calculated_ni - net_income) # Difference should be small (other income/expense) acceptable_diff = abs(revenue) * 0.1 # 10% tolerance for other items if difference > acceptable_diff: print(f"WARNING: Income statement doesn't reconcile:") print(f" Revenue: {revenue:,.0f}") print(f" Total Expenses: {total_expenses:,.0f}") print(f" Calculated NI: {calculated_ni:,.0f}") print(f" Reported NI: {net_income:,.0f}") print(f" Difference: {difference:,.0f} (acceptable: {acceptable_diff:,.0f})") return False return True` ### Unmapped Tag Detection Detect XBRL tags that aren't mapped to standard concepts: `def find_unmapped_tags(xbrl, mapper): """Find all XBRL tags that don't map to standard concepts.""" unmapped = [] # Get all unique concepts facts = xbrl.facts.query().to_dataframe() concepts = facts['concept'].unique() for concept in concepts: # Try to map each concept label = facts[facts['concept'] == concept]['label'].iloc[0] context = {'statement_type': 'Unknown'} standard_concept = mapper.map_concept(concept, label, context) if standard_concept is None: unmapped.append({ 'concept': concept, 'label': label, 'occurrences': len(facts[facts['concept'] == concept]) }) # Sort by occurrences (most common first) unmapped.sort(key=lambda x: x['occurrences'], reverse=True) return unmapped # Usage from edgar.xbrl.standardization import MappingStore, ConceptMapper store = MappingStore() mapper = ConceptMapper(store) unmapped = find_unmapped_tags(xbrl, mapper) print(f"Found {len(unmapped)} unmapped tags") for tag in unmapped[:10]: # Top 10 print(f" {tag['concept']}: '{tag['label']}' ({tag['occurrences']} occurrences)")` ### Logging Unmapped Tags for Review Create a log file of unmapped tags with suggested mappings: `import csv from difflib import get_close_matches def log_unmapped_tags(xbrl, mapper, output_path='unmapped_tags.csv'): """Create CSV log of unmapped tags with suggested standard concepts.""" unmapped = find_unmapped_tags(xbrl, mapper) # Get all standard concepts for matching standard_concepts = list(store.mappings.keys()) with open(output_path, 'w', newline='') as f: writer = csv.writer(f) writer.writerow([ 'company_concept', 'label', 'occurrences', 'suggested_mapping', 'confidence', 'cik', 'notes' ]) for tag in unmapped: # Find closest matching standard concept matches = get_close_matches( tag['label'], standard_concepts, n=1, cutoff=0.6 ) suggested = matches[0] if matches else "MANUAL_REVIEW_NEEDED" confidence = "high" if matches and len(matches[0]) else "low" writer.writerow([ tag['concept'], tag['label'], tag['occurrences'], suggested, confidence, xbrl.entity_identifier, "" # Manual notes column ]) print(f"Wrote {len(unmapped)} unmapped tags to {output_path}") print("Review file and add to concept_mappings.json") # Usage - process all companies companies = ["AAPL", "MSFT", "GOOGL", "TSLA"] for ticker in companies: company = Company(ticker) filing = company.get_filings(form="10-K").latest() xbrl = filing.xbrl() log_unmapped_tags(xbrl, mapper, f"unmapped_{ticker}.csv")` ### Validation Utility Script Comprehensive validation script for batch processing: `def validate_company_mappings(ticker, form="10-K", years=3): """Validate mappings for a company across multiple years.""" company = Company(ticker) filings = company.get_filings(form=form).head(years) results = [] for filing in filings: print(f"\nValidating {ticker} {filing.filing_date}...") try: xbrl = filing.xbrl() period_key = xbrl.reporting_periods[0]['key'] # Run validations bs_valid = validate_balance_sheet(xbrl, period_key) is_valid = validate_income_statement(xbrl, period_key) unmapped = find_unmapped_tags(xbrl, mapper) result = { 'ticker': ticker, 'filing_date': filing.filing_date, 'balance_sheet_valid': bs_valid, 'income_statement_valid': is_valid, 'unmapped_count': len(unmapped), 'total_concepts': len(xbrl.facts.query().to_dataframe()['concept'].unique()) } results.append(result) print(f" Balance Sheet: {'✓' if bs_valid else '✗'}") print(f" Income Statement: {'✓' if is_valid else '✗'}") print(f" Unmapped: {len(unmapped)}") except Exception as e: print(f" ERROR: {e}") results.append({ 'ticker': ticker, 'filing_date': filing.filing_date, 'error': str(e) }) return results # Batch validation companies = ["AAPL", "MSFT", "GOOGL", "TSLA", "AMZN"] all_results = [] for ticker in companies: results = validate_company_mappings(ticker) all_results.extend(results) # Summary valid_count = sum(1 for r in all_results if r.get('balance_sheet_valid', False)) print(f"\nOverall: {valid_count}/{len(all_results)} filings validated successfully")` * * * 8\. CSV Workflow ---------------- While EdgarTools currently uses JSON for mappings, many users prefer CSV for editing. This section provides utilities for CSV-based workflows. **Note**: Native CSV support is planned for v4.29.0/v4.30.0. ### Why CSV for Mapping Management? **Advantages**: - **Excel editing**: Use familiar spreadsheet tools - **Duplicate detection**: Sort columns to find duplicates easily - **Filtering**: Quick filtering by standard concept or company - **Validation**: Formulas can check for consistency - **Collaboration**: Easier for non-technical team members ### CSV Format Specification **Standard Format**: `standard_concept,company_concept,company_cik,priority,notes Revenue,us-gaap:Revenue,,1,Core GAAP concept Revenue,us-gaap:Revenues,,1,Alternative spelling Automotive Revenue,tsla:AutomotiveRevenue,1318605,2,Tesla-specific Automotive Revenue,tsla:VehicleRevenue,1318605,2,Alternative Tesla tag` **Columns**: - `standard_concept`: The standardized label (e.g., "Revenue") - `company_concept`: The XBRL tag (e.g., "us-gaap:Revenue") - `company_cik`: Optional CIK for company-specific mappings (empty for core) - `priority`: 1=core, 2=company-specific (optional, for reference) - `notes`: Explanation, context, or validation notes ### Export Mappings to CSV `import csv from edgar.xbrl.standardization import MappingStore def export_mappings_to_csv(store: MappingStore, output_path: str): """Export MappingStore to CSV format for editing.""" rows = [] # Export core mappings (priority 1) for standard_concept, company_concepts in store.mappings.items(): for company_concept in company_concepts: rows.append({ 'standard_concept': standard_concept, 'company_concept': company_concept, 'company_cik': '', 'priority': 1, 'notes': 'Core mapping' }) # Export company-specific mappings (priority 2) for entity_id, company_data in store.company_mappings.items(): cik = company_data.get('metadata', {}).get('cik', '') concept_mappings = company_data.get('concept_mappings', {}) for standard_concept, company_concepts in concept_mappings.items(): for company_concept in company_concepts: rows.append({ 'standard_concept': standard_concept, 'company_concept': company_concept, 'company_cik': cik, 'priority': 2, 'notes': f'Company-specific: {entity_id}' }) # Write to CSV with open(output_path, 'w', newline='') as f: fieldnames = ['standard_concept', 'company_concept', 'company_cik', 'priority', 'notes'] writer = csv.DictWriter(f, fieldnames=fieldnames) writer.writeheader() writer.writerows(rows) print(f"Exported {len(rows)} mappings to {output_path}") # Usage store = MappingStore() export_mappings_to_csv(store, 'all_mappings.csv')` ### Import Mappings from CSV `import csv from collections import defaultdict import json def import_mappings_from_csv(csv_path: str): """Import mappings from CSV and generate JSON files.""" core_mappings = defaultdict(list) company_mappings = defaultdict(lambda: defaultdict(list)) with open(csv_path, 'r') as f: reader = csv.DictReader(f) for row in reader: standard_concept = row['standard_concept'] company_concept = row['company_concept'] cik = row.get('company_cik', '').strip() if cik: # Company-specific mapping company_mappings[cik][standard_concept].append(company_concept) else: # Core mapping core_mappings[standard_concept].append(company_concept) # Save core mappings with open('concept_mappings.json', 'w') as f: json.dump(dict(core_mappings), f, indent=2) print(f"Saved core mappings: {len(core_mappings)} concepts") # Save company-specific mappings for cik, mappings in company_mappings.items(): # Find ticker from CIK (simplified - you'd need a CIK-to-ticker lookup) ticker = f"cik{cik}" # Placeholder company_data = { "metadata": { "entity_identifier": ticker, "cik": cik, "priority": "high", "created_date": "2025-11-19" }, "concept_mappings": dict(mappings) } filename = f"{ticker}_mappings.json" with open(filename, 'w') as f: json.dump(company_data, f, indent=2) print(f"Saved company mappings: {filename}") # Usage import_mappings_from_csv('all_mappings.csv')` ### Excel Editing Workflow **Step 1: Export to CSV** `from edgar.xbrl.standardization import MappingStore store = MappingStore() export_mappings_to_csv(store, 'edgartools_mappings.csv')` **Step 2: Open in Excel** - Open `edgartools_mappings.csv` in Excel or Google Sheets - Use Excel features: - **Sort** by `standard_concept` to group related mappings - **Filter** by `company_cik` to see company-specific mappings - **Conditional Formatting** to highlight duplicates - **Find & Replace** for bulk updates **Step 3: Duplicate Detection in Excel** Formula in column F (next to your data): `=COUNTIFS($B:$B,B2,$A:$A,A2)>1` This highlights if the same `company_concept` maps to the same `standard_concept` multiple times. **Step 4: Validation in Excel** Add a validation column with this formula: `=IF(ISBLANK(B2), "Missing concept", IF(ISBLANK(A2), "Missing label", IF(AND(C2<>"", NOT(ISNUMBER(C2))), "Invalid CIK", "OK")))` **Step 5: Import Back to JSON** `import_mappings_from_csv('edgartools_mappings.csv')` ### Single File vs Multiple Files Two approaches for managing 200+ companies: #### Approach 1: Single CSV File (Recommended for Excel Users) **Structure**: `standard_concept,company_concept,company_cik,ticker,notes Revenue,us-gaap:Revenue,,,Core GAAP Automotive Revenue,tsla:AutomotiveRevenue,1318605,TSLA,Tesla-specific Energy Revenue,tsla:EnergyRevenue,1318605,TSLA,Tesla energy Product Revenue,msft:ProductRevenue,789019,MSFT,Microsoft` **Advantages**: - Easy to search across all companies - Single source of truth - Easy duplicate detection - Better for bulk operations **Disadvantages**: - Large file size (200 companies = 10,000+ rows) - Merge conflicts in version control - Slower to load #### Approach 2: Multiple JSON Files (Current EdgarTools Approach) **Structure**: `company_mappings/ aapl_mappings.json msft_mappings.json tsla_mappings.json googl_mappings.json ...` **Advantages**: - Modular (edit one company at a time) - Better for version control (fewer merge conflicts) - Faster loading (only load relevant companies) - Clear ownership (one file per company) **Disadvantages**: - Harder to find duplicates across companies - More files to manage - Need tooling to search across all files #### Hybrid Approach (Best of Both Worlds) Use CSV as master source, generate JSON files: `def csv_to_company_json_files(csv_path: str, output_dir: str): """Convert single CSV to multiple company JSON files.""" import csv import json from pathlib import Path from collections import defaultdict Path(output_dir).mkdir(exist_ok=True) # Group by CIK company_data = defaultdict(lambda: { 'metadata': {}, 'concept_mappings': defaultdict(list) }) with open(csv_path, 'r') as f: reader = csv.DictReader(f) for row in reader: cik = row.get('company_cik', '').strip() if not cik: continue # Skip core mappings ticker = row.get('ticker', f'cik{cik}').lower() # Set metadata if not company_data[ticker]['metadata']: company_data[ticker]['metadata'] = { 'entity_identifier': ticker, 'cik': cik, 'priority': 'high' } # Add mapping standard = row['standard_concept'] concept = row['company_concept'] company_data[ticker]['concept_mappings'][standard].append(concept) # Write files for ticker, data in company_data.items(): # Convert defaultdict to regular dict data['concept_mappings'] = dict(data['concept_mappings']) filename = Path(output_dir) / f"{ticker}_mappings.json" with open(filename, 'w') as f: json.dump(data, f, indent=2) concept_count = len(data['concept_mappings']) print(f"Created {filename} with {concept_count} concepts") # Usage csv_to_company_json_files( 'master_mappings.csv', 'company_mappings/' )` **Recommended Workflow for 200+ Companies**: 1. Maintain master CSV file: `edgartools_master_mappings.csv` 2. Edit in Excel (easy duplicate detection, filtering) 3. Run conversion script to generate JSON files 4. Deploy JSON files to package directory 5. Version control both CSV (master) and JSON (generated) * * * 9\. Real-World Examples ----------------------- This section explains existing company mapping files with detailed annotations. ### Example 1: Tesla (Automotive + Energy) Tesla has a complex revenue structure combining automotive sales, leasing, and energy generation/storage. **File**: `company_mappings/tsla_mappings.json` `{ "metadata": { "entity_identifier": "tsla", "company_name": "Tesla, Inc.", "cik": "1318605", "priority": "high", "created_date": "2024-06-25", "last_updated": "2024-06-25", "description": "Tesla-specific concept mappings to handle automotive, energy, and service revenue streams" }, "concept_mappings": { "Automotive Revenue": [ "tsla:AutomotiveRevenue", "tsla:AutomotiveSales", "tsla:VehicleRevenue" ], "Automotive Leasing Revenue": [ "tsla:AutomotiveLeasing", "tsla:AutomotiveLeasingRevenue", "tsla:VehicleLeasingRevenue" ], "Energy Revenue": [ "tsla:EnergyGenerationAndStorageRevenue", "tsla:EnergyRevenue", "tsla:SolarRevenue", "tsla:EnergyStorageRevenue" ], "Service Revenue": [ "tsla:ServicesAndOtherRevenue", "tsla:ServiceRevenue", "tsla:SuperchargerRevenue" ] }, "hierarchy_rules": { "Revenue": { "children": [ "Automotive Revenue", "Energy Revenue", "Service Revenue" ] }, "Automotive Revenue": { "children": [ "Automotive Leasing Revenue" ] } }, "business_context": { "primary_revenue_streams": ["automotive", "energy", "services"], "revenue_model": "product_and_service", "key_metrics": ["vehicle_deliveries", "energy_deployments"], "industry": "automotive_technology" } }` **Key Design Decisions**: 1. **Granular Revenue Breakdown**: 2. Separate automotive sales from leasing (different economics) 3. Distinguish energy from automotive (different growth drivers) 4. Services as distinct category (recurring revenue) 5. **Hierarchy Rules**: 6. `Revenue` is parent of three main streams 7. `Automotive Revenue` contains `Automotive Leasing Revenue` as child 8. This ensures proper nesting in financial statements 9. **Multiple Tag Variations**: 10. Tesla has changed tag names over time (`AutomotiveRevenue` vs `AutomotiveSales`) 11. All variations map to same standard concept for consistency **Usage Example**: `from edgar import Company tesla = Company("TSLA") filing = tesla.get_filings(form="10-K").latest() xbrl = filing.xbrl() # Get standardized income statement income = xbrl.statements.income_statement() # Tesla-specific revenue line items will appear as: # - Automotive Revenue (instead of generic "Revenue") # - Automotive Leasing Revenue # - Energy Revenue # - Service Revenue` ### Example 2: Microsoft (Technology Platform) Microsoft has platform-based revenue (Azure, Office 365, Dynamics) requiring specialized mapping. **File**: `company_mappings/msft_mappings.json` `{ "entity_info": { "name": "Microsoft Corporation", "cik": "0000789019", "ticker": "MSFT", "description": "Microsoft-specific concept mappings for unique business terminology" }, "concept_mappings": { "_comment_msft_revenue": "Microsoft uses specific revenue categorization that differs from standard tech companies", "Product Revenue": [ "msft:ProductRevenue", "msft:WindowsCommercialRevenue", "msft:WindowsConsumerRevenue", "msft:OfficeCommercialRevenue" ], "Service Revenue": [ "msft:ServiceRevenue", "msft:CloudServicesRevenue", "msft:ConsultingServicesRevenue" ], "Subscription Revenue": [ "msft:Office365CommercialRevenue", "msft:Office365ConsumerRevenue", "msft:DynamicsRevenue" ], "Platform Revenue": [ "msft:AzureRevenue", "msft:XboxContentAndServicesRevenue" ], "_comment_msft_expenses": "Microsoft has unique expense categorizations for sales and marketing vs G&A", "Sales and Marketing Expense": [ "msft:SalesAndMarketingExpense", "msft:AdvertisingAndPromotionExpense" ], "Technical Support Expense": [ "msft:TechnicalSupportExpense", "msft:CustomerSupportExpense" ] }, "hierarchy_rules": { "_comment": "Rules for handling Microsoft-specific hierarchical relationships", "revenue_hierarchy": { "parent": "Revenue", "children": ["Product Revenue", "Service Revenue", "Subscription Revenue", "Platform Revenue"], "calculation_rule": "sum" }, "expense_hierarchy": { "parent": "Operating Expenses", "children": ["Sales and Marketing Expense", "Technical Support Expense"], "calculation_rule": "sum" } } }` **Key Design Decisions**: 1. **Four Revenue Categories**: 2. **Product**: Traditional software sales (Windows, Office perpetual licenses) 3. **Service**: Consulting, support services 4. **Subscription**: Recurring revenue (Office 365, Dynamics) 5. **Platform**: Cloud platforms (Azure, Xbox services) 6. **Expense Granularity**: 7. Separates sales/marketing from technical support 8. Reflects Microsoft's investment in customer success teams 9. **Hierarchy Rules with Calculation**: 10. Explicit `calculation_rule: sum` indicates children should sum to parent 11. Validation can check this relationship **Usage Example**: `msft = Company("MSFT") filing = msft.get_filings(form="10-K").latest() xbrl = filing.xbrl() # Analyze revenue mix facts = xbrl.facts.query().by_statement_type("IncomeStatement").to_dataframe() revenue_breakdown = facts[facts['label'].str.contains('Revenue')][['label', 'value']] print(revenue_breakdown) # Output: # label value # Product Revenue 75,000,000,000 # Service Revenue 25,000,000,000 # Subscription Revenue 60,000,000,000 # Platform Revenue 40,000,000,000 # Revenue 200,000,000,000` ### Example 3: Berkshire Hathaway (Conglomerate) Berkshire Hathaway is a diversified holding company with insurance, utilities, railroads, and manufacturing. **File**: `company_mappings/brka_mappings.json` `{ "concept_mappings": { "Sales and Service Revenue": [ "brka:SalesAndServiceRevenue" ] }, "hierarchy_rules": { "Revenue": { "components": [ "Sales and Service Revenue", "Operating Lease Revenue" ], "description": "Total revenue comprises sales/service revenue and operating lease income for holding company" } }, "business_context": { "entity_type": "holding_company", "industry": "diversified_conglomerate", "description": "Berkshire Hathaway operates diverse businesses including insurance, utilities, railroads, and manufacturing" } }` **Key Design Decisions**: 1. **Minimal Customization**: 2. Berkshire uses mostly standard US-GAAP tags 3. Only needs mapping for unique revenue categorization 4. **Lease Revenue Separation**: 5. Operating lease revenue (equipment leasing subsidiaries) 6. Separated from core sales/service revenue 7. **Business Context**: 8. Documents the holding company structure 9. Helps interpreters understand diverse revenue sources **Why So Simple?**: - Berkshire's filings primarily use standard US-GAAP taxonomy - Conglomerates often don't need extensive custom tags - Industry-specific tags are used by individual subsidiaries (not parent) ### Example 4: Industrial Company Template For users managing 200+ industrial companies, here's a template: `{ "metadata": { "entity_identifier": "ticker", "company_name": "Company Name", "cik": "0000000000", "priority": "medium", "created_date": "2025-11-19", "last_updated": "2025-11-19", "description": "Industrial company with manufacturing operations", "industry": "industrial_manufacturing" }, "concept_mappings": { "_comment": "Common industrial company customizations", "Product Sales": [ "company:ProductSales", "company:ManufacturedGoodsSales" ], "Raw Materials Inventory": [ "company:RawMaterialsInventory" ], "Work in Process Inventory": [ "company:WorkInProcessInventory" ], "Finished Goods Inventory": [ "company:FinishedGoodsInventory" ], "Manufacturing Overhead": [ "company:ManufacturingOverhead", "company:FactoryOverhead" ] }, "hierarchy_rules": { "Inventory": { "children": [ "Raw Materials Inventory", "Work in Process Inventory", "Finished Goods Inventory" ], "calculation_rule": "sum", "description": "Manufacturing inventory breakdown" } }, "business_context": { "primary_revenue_streams": ["product_sales"], "revenue_model": "manufacturing", "key_metrics": ["inventory_turnover", "production_efficiency"], "industry": "industrial_manufacturing", "notes": "Focus on inventory management and cost of goods sold structure" } }` **Adaptation for Your Companies**: 1. Copy this template 2. Replace `company:` prefix with actual company prefix 3. Add industry-specific concepts (automotive parts, chemicals, etc.) 4. Customize inventory structure based on business model * * * 10\. Future Enhancements ------------------------ This section outlines the roadmap for standardization improvements based on user feedback. ### Version 4.30.0 (Next 1-2 Months) **Focus**: Configuration and CSV Support #### 1\. Configurable Mapping Paths **Problem Solved**: Users can maintain mappings outside package directory. **Implementation**: `# Environment variable configuration import os os.environ['EDGAR_CORE_MAPPINGS'] = '/path/to/my/concept_mappings.json' os.environ['EDGAR_COMPANY_MAPPINGS_DIR'] = '/path/to/my/company_mappings/' # Library loads from custom paths from edgar.xbrl.standardization import MappingStore store = MappingStore() # Automatically uses env var paths` **Alternative: Constructor parameters**: `store = MappingStore( core_mappings_path='/path/to/concept_mappings.json', company_mappings_dir='/path/to/company_mappings/' )` **Benefits**: - Separate version control for mappings - Mappings survive package upgrades - Multiple mapping sets for different use cases #### 2\. Native CSV Format Support **Problem Solved**: Excel-based workflows without conversion scripts. **Implementation**: `# Auto-detect format from extension store = MappingStore(core_mappings_path='my_mappings.csv') # Explicit format specification store = MappingStore( core_mappings_path='my_mappings.txt', format='csv' )` **CSV Format**: `standard_concept,company_concept,notes Revenue,us-gaap:Revenue,Core GAAP tag Revenue,us-gaap:Revenues,Alternative spelling` **Benefits**: - No conversion scripts needed - Direct Excel editing - Easier duplicate detection #### 3\. Enhanced Validation Tools **Problem Solved**: Automated mapping quality checks. **Implementation**: `from edgar.xbrl.standardization import MappingValidator validator = MappingValidator(store) # Validate balance sheet balancing report = validator.validate_company( ticker="AAPL", form="10-K", years=3 ) print(report.summary()) # Output: # ✓ Balance Sheet: 3/3 periods balanced # ✓ Income Statement: 3/3 periods validated # ⚠ Unmapped tags: 12 concepts need mapping` **Features**: - Batch validation across multiple companies - Balance sheet equation checking - Income statement reconciliation - Coverage reports (% of concepts mapped) ### Version 4.31.0 (2-3 Months) **Focus**: CIK-Based Identification #### 1\. CIK as Primary Identifier **Problem Solved**: Handle multi-ticker companies (GOOG/GOOGL, HEI.A/HEI.B). **Implementation**: `{ "metadata": { "entity_identifier": "0001652044", "cik": "0001652044", "tickers": ["GOOG", "GOOGL"], "primary_ticker": "GOOG", "company_name": "Alphabet Inc." } }` **File Naming**: `company_mappings/ cik0001652044_mappings.json # CIK-based naming # OR legacy support: goog_mappings.json # Ticker-based naming (still supported)` #### 2\. Dual Lookup Support **During Transition**: Support both ticker and CIK lookups. `# Both work store.get_company_mappings(ticker="GOOG") store.get_company_mappings(cik="0001652044")` #### 3\. Migration Tool **Help users migrate** from ticker-based to CIK-based files. `from edgar.xbrl.standardization import migrate_to_cik # Migrate all ticker-based files to CIK-based migrate_to_cik( input_dir='company_mappings/', output_dir='company_mappings_cik/', cik_lookup_file='ticker_to_cik.csv' )` ### Version 5.0.0 (Major Release) **Focus**: Advanced Features and ML Integration #### 1\. JSON-Loaded StandardConcept **Problem Solved**: StandardConcept enum becomes fully data-driven. **Current**: `# Enum is hardcoded in Python class StandardConcept(str, Enum): REVENUE = "Revenue" TOTAL_ASSETS = "Total Assets"` **Future**: `# Enum loaded from JSON at runtime StandardConcept = load_concepts_from_json('standard_concepts.json') # Users can extend without touching Python code` #### 2\. Concept Marketplace/Repository **Problem Solved**: Share mappings across community. **Vision**: `from edgar.xbrl.standardization import ConceptMarketplace marketplace = ConceptMarketplace() # Download community mappings marketplace.install('industrial-companies-pack') marketplace.install('tech-companies-pack') # Share your mappings marketplace.publish( 'my-custom-mappings', description='Custom mappings for 200+ industrial firms', companies=['AAPL', 'MSFT', ...], license='MIT' )` **Features**: - Community-contributed mappings - Rating and review system - Automatic updates - Industry-specific packs #### 3\. ML-Based Concept Inference **Problem Solved**: Automatically suggest mappings for unmapped tags. **Implementation**: `from edgar.xbrl.standardization import MLConceptMapper ml_mapper = MLConceptMapper() # Train on existing mappings ml_mapper.train(store.mappings) # Suggest mappings for unmapped concepts suggestion = ml_mapper.suggest( concept='company:CustomRevenueConcept', label='Sales of Manufactured Goods', context={'statement_type': 'IncomeStatement'} ) print(suggestion) # Output: # Suggested: "Product Revenue" # Confidence: 0.89 # Similar concepts: ["Revenue", "Product Sales", "Sales"]` **Features**: - Learn from existing mappings - Context-aware suggestions - Confidence scoring - Interactive review workflow #### 4\. Advanced Validation Framework **Problem Solved**: Comprehensive statement validation. `from edgar.xbrl.standardization import ValidationFramework framework = ValidationFramework(store) # Define custom validation rules @framework.rule(statement='BalanceSheet', severity='error') def validate_accounting_equation(facts): assets = facts.get('Total Assets') liabilities = facts.get('Total Liabilities') equity = facts.get("Total Stockholders' Equity") if abs(assets - (liabilities + equity)) > 1.0: return ValidationError("Accounting equation violated") return None # Run validation results = framework.validate_company('AAPL', years=10) results.generate_report('validation_report.html')` ### Timeline Summary | Feature | Version | Timeline | Status | | --- | --- | --- | --- | | Configurable paths | v4.30.0 | 1-2 months | Planned | | Native CSV support | v4.30.0 | 1-2 months | Planned | | Enhanced validation | v4.30.0 | 1-2 months | Planned | | CIK-based identification | v4.31.0 | 2-3 months | Planned | | Dual lookup support | v4.31.0 | 2-3 months | Planned | | Migration tool | v4.31.0 | 2-3 months | Planned | | JSON StandardConcept | v5.0.0 | 6-12 months | Under consideration | | Concept marketplace | v5.0.0 | 6-12 months | Under consideration | | ML concept inference | v5.0.0 | 6-12 months | Research phase | ### Providing Feedback Your feedback shapes these enhancements. To contribute: 1. **GitHub Issues**: Comment on issue #494 or create new issues 2. **Feature Requests**: Use the feature request template 3. **User Stories**: Share your specific use cases 4. **Beta Testing**: Volunteer to test pre-release versions **Contact**: - GitHub: https://github.com/dgunning/edgartools/issues/494 - Discussions: https://github.com/dgunning/edgartools/discussions * * * Summary and Quick Reference --------------------------- ### When to Customize Standardization ✅ **Yes, customize when**: - Managing 200+ companies with diverse taxonomies - Industry-specific valuations (industrial, automotive, tech) - Building models requiring consistent data structure - Statement balancing is critical to your workflow ❌ **No, use defaults when**: - Analyzing 1-10 companies - Standard US-GAAP concepts are sufficient - Quick analysis or exploration - Don't need custom taxonomy support ### Quick Decision Tree `Do you analyze 200+ companies? ├─ Yes → Use custom company-specific mappings (Section 4) │ └─ CSV workflow for easier management (Section 8) └─ No → Do you need industry-specific concepts? ├─ Yes → Use custom core mappings (Section 3) └─ No → Use default StandardConcept mappings` ### Essential Resources | Task | Section | Key File | | --- | --- | --- | | Understand architecture | Section 2 | `core.py` | | Add core mappings | Section 3 | `concept_mappings.json` | | Create company mappings | Section 4 | `{ticker}_mappings.json` | | Resolve ambiguous tags | Section 5 | Your context analysis | | Work around limitations | Section 6 | Installation scripts | | Validate mappings | Section 7 | Validation utilities | | Use CSV workflow | Section 8 | CSV utilities | | Learn from examples | Section 9 | Tesla, Microsoft files | ### Key Concepts Clarified | Concept | What It Is | What It's NOT | | --- | --- | --- | | **StandardConcept Enum** | IDE convenience, type safety | NOT the mapping data | | **JSON Mappings** | Source of truth for mappings | NOT just for reference | | **Priority System** | Conflict resolution | NOT just ordering | | **CIK** | Stable company identifier | NOT ticker (which changes) | | **Context** | Ambiguity resolution | NOT just metadata | ### Contact and Support * **GitHub Issue**: #494 * **Documentation**: This guide * **Examples**: Section 9 * **Roadmap**: Section 10 * * * **Document Version**: 1.0 **Last Updated**: 2025-11-19 **EdgarTools Version**: 4.29.0+ **Contributors**: @dgunning, @mpreiss9, EdgarTools community Back to top --- # Customizing Standardization - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/advanced/customizing-standardization/#customizing-xbrl-standardization) Customizing XBRL Standardization ================================ **Target Audience**: Advanced users, financial analysts, quantitative researchers **Prerequisites**: Understanding of XBRL concepts, Python basics, JSON format **Use Case**: Custom taxonomies, 200+ companies, industry-specific valuations * * * Table of Contents ----------------- 1. [Overview and Introduction](https://edgartools.readthedocs.io/en/stable/advanced/customizing-standardization/#1-overview-and-introduction) 2. [Architecture and Design](https://edgartools.readthedocs.io/en/stable/advanced/customizing-standardization/#2-architecture-and-design) 3. [Core Mappings Structure](https://edgartools.readthedocs.io/en/stable/advanced/customizing-standardization/#3-core-mappings-structure) 4. [Company-Specific Mappings](https://edgartools.readthedocs.io/en/stable/advanced/customizing-standardization/#4-company-specific-mappings) 5. [Priority System and Ambiguous Tag Resolution](https://edgartools.readthedocs.io/en/stable/advanced/customizing-standardization/#5-priority-system-and-ambiguous-tag-resolution) 6. [Current Limitations](https://edgartools.readthedocs.io/en/stable/advanced/customizing-standardization/#6-current-limitations) 7. [Validation Techniques](https://edgartools.readthedocs.io/en/stable/advanced/customizing-standardization/#7-validation-techniques) 8. [CSV Workflow](https://edgartools.readthedocs.io/en/stable/advanced/customizing-standardization/#8-csv-workflow) 9. [Real-World Examples](https://edgartools.readthedocs.io/en/stable/advanced/customizing-standardization/#9-real-world-examples) 10. [Future Enhancements](https://edgartools.readthedocs.io/en/stable/advanced/customizing-standardization/#10-future-enhancements) * * * 1\. Overview and Introduction ----------------------------- ### What is XBRL Standardization? XBRL standardization is the process of mapping company-specific XBRL tags to a consistent set of standardized concept names. This enables: * **Consistent presentation** of financial statements across different companies * **Comparable analysis** regardless of each company's unique taxonomy * **Automated processing** of financial data from diverse sources * **Reduced complexity** when working with 200+ companies ### Why Companies Need Custom Taxonomies Every company's XBRL filing uses a mix of: - **US-GAAP standard tags**: `us-gaap:Revenue`, `us-gaap:Assets` - **Company-specific extensions**: `tsla:AutomotiveRevenue`, `msft:AzureRevenue` - **Industry-specific concepts**: Energy, automotive, technology sectors **The Problem**: Without standardization, analyzing 200 companies means dealing with thousands of unique XBRL tag variations for the same underlying financial concept. **The Solution**: EdgarTools' standardization system maps all these variations to a unified set of standard concepts. ### When to Customize Standardization You should customize the standardization system when: * **Managing 200+ companies** with diverse taxonomies * **Working with industry-specific filings** (automotive, technology, industrial firms) * **Building valuation models** requiring granular financial statements * **Conducting multi-company analysis** that requires consistent data structure * **Ensuring statement balancing** (Assets = Liabilities + Equity) across diverse filings ### What This Guide Covers This comprehensive guide explains: - How the standardization architecture works - How to create custom concept mappings - How to handle ambiguous XBRL tags (200+ identified cases) - How to validate mapping quality - Production-ready workflows for managing custom taxonomies * * * 2\. Architecture and Design --------------------------- ### The StandardConcept Enum vs JSON Mappings This is a critical distinction that causes confusion: #### StandardConcept Enum: IDE Convenience (Optional) `from edgar.xbrl.standardization import StandardConcept # Enum provides autocomplete and type safety revenue_label = StandardConcept.REVENUE.value # "Revenue" assets_label = StandardConcept.TOTAL_ASSETS.value # "Total Assets"` **Purpose**: - IDE autocomplete for known concepts - Type safety for Python code - Semantic meaning for core financial concepts **Location**: `edgar/xbrl/standardization/core.py` (lines 18-126) #### JSON Mappings: Source of Truth (Required) `{ "Revenue": [ "us-gaap:Revenue", "us-gaap:Revenues", "us-gaap:SalesRevenueNet" ] }` **Purpose**: - The actual mapping data used by the system - Unlimited extensibility without code changes - User-customizable without touching Python code **Location**: `edgar/xbrl/standardization/concept_mappings.json` #### Critical Clarification **The Relationship**: - Enum values SHOULD match JSON keys (e.g., `StandardConcept.REVENUE.value == "Revenue"`) - This relationship is NOT enforced by code - JSON is what the system actually uses for mapping - Enum is purely for developer convenience **For Custom Mappings**: - **You customize via JSON files** - NOT by modifying the enum - The enum can remain unchanged; JSON drives all behavior - You can add mappings in JSON that don't exist in the enum - System will validate JSON keys against enum if you enable validation (optional) ### How Standardization Works The standardization system follows this flow: `Company XBRL Tag → MappingStore → Priority Resolution → Standard Concept` **Example**: `"tsla:AutomotiveRevenue" → [Priority 4: Tesla mapping] → "Automotive Revenue" "us-gaap:Revenue" → [Priority 1: Core mapping] → "Revenue"` ### Key Components #### 1\. MappingStore (The Brain) **File**: `edgar/xbrl/standardization/core.py` (lines 128-462) **Responsibilities**: - Loads core mappings from `concept_mappings.json` - Loads company-specific mappings from `company_mappings/` directory - Merges mappings with priority scoring - Resolves ambiguous tags using context **Initialization**: `from edgar.xbrl.standardization import MappingStore # Default initialization (loads packaged mappings) store = MappingStore() # Custom source (future enhancement) store = MappingStore(source="/path/to/custom_mappings.json") # Read-only mode (for testing) store = MappingStore(read_only=True)` #### 2\. ConceptMapper (The Worker) **File**: `edgar/xbrl/standardization/core.py` (lines 464-682) **Responsibilities**: - Maps individual concepts using MappingStore - Caches results for performance - Handles context-aware inference - Tracks unmapped concepts **Usage**: `mapper = ConceptMapper(mapping_store) # Map a concept with context context = { 'statement_type': 'BalanceSheet', 'level': 0, 'is_total': True } standard_concept = mapper.map_concept( company_concept='us-gaap:Assets', label='Total Assets', context=context ) # Returns: "Total Assets"` #### 3\. Priority System The system uses priority levels to resolve conflicts: | Priority | Source | Description | Example | | --- | --- | --- | --- | | **P1** | Core mappings | Base US-GAAP concepts | `us-gaap:Revenue → "Revenue"` | | **P2** | Company mappings | Company-specific overrides | `tsla:Revenue → "Automotive Revenue"` | | **P4** | Detected entity | Auto-detected from prefix | `tsla:CustomTag → uses Tesla P2 mappings` | **Priority Resolution Algorithm** (lines 408-449): 1. Detect entity from concept prefix (e.g., `tsla:` → `"tsla"`) 2. Search through merged mappings 3. For each match, calculate effective priority 4. If detected entity matches mapping source, boost to P4 5. Return highest priority match * * * 3\. Core Mappings Structure --------------------------- ### File Location **Current Location** (hardcoded): `edgar/xbrl/standardization/concept_mappings.json` **Future Enhancement** (v4.30.0): Configurable paths via environment variables. ### JSON Structure The core mappings file uses a flat dictionary structure: `{ "Standard Concept Label": [ "company_specific_tag_1", "company_specific_tag_2", "us-gaap:StandardTag" ], "_comment_section": "Documentation comments for maintainers" }` ### Real Example from concept\_mappings.json `{ "_comment_revenue_hierarchy": "REVENUE HIERARCHY FIX: Separated total revenue from component revenue types to prevent duplicate labels.", "Revenue": [ "us-gaap:Revenue", "us-gaap:Revenues", "us-gaap:SalesRevenueNet", "us-gaap:OperatingRevenue" ], "Contract Revenue": [ "us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax", "us-gaap:RevenueFromContractWithCustomerIncludingAssessedTax" ], "Product Revenue": [ "us-gaap:SalesRevenueGoodsNet", "us-gaap:ProductSales" ] }` ### Understanding Comments The JSON file includes `_comment_*` keys for documentation: - These are ignored by the mapping system - They explain design decisions - They help maintainers understand complex hierarchies ### Hierarchy Separation Notice the careful separation of concepts: - **"Revenue"**: Total revenue (parent concept) - **"Contract Revenue"**: Component of revenue (child concept) - **"Product Revenue"**: Another component (sibling to Contract Revenue) This prevents mapping conflicts where multiple XBRL tags map to the same label. ### Cost Hierarchy Example `{ "_comment_cost_of_revenue_hierarchy": "Different business models use different cost concepts that should have distinct labels.", "Total Cost of Revenue": [ "us-gaap:CostOfRevenue" ], "Cost of Goods Sold": [ "us-gaap:CostOfGoodsSold" ], "Cost of Goods and Services Sold": [ "us-gaap:CostOfGoodsAndServicesSold" ], "Direct Operating Costs": [ "us-gaap:DirectOperatingCosts" ] }` **Why separate these?** - Manufacturing companies use "Cost of Goods Sold" - Service companies use "Direct Operating Costs" - Mixed businesses use "Cost of Goods and Services Sold" - Each should have a distinct label for clarity ### Adding Custom Core Mappings To extend core mappings (not recommended for most users): 1. **Locate the file**: `edgar/xbrl/standardization/concept_mappings.json` 2. **Add your mapping**: `{ "Custom Concept Label": [ "us-gaap:YourCustomTag", "company:AnotherTag" ] }` 3. **Maintain hierarchy**: Ensure parent/child relationships are clear 4. **Add comments**: Document your reasoning with `_comment_*` keys **Warning**: Modifying packaged files is not recommended. Use company-specific mappings instead (Section 4). * * * 4\. Company-Specific Mappings ----------------------------- ### Why Company-Specific Mappings? Company-specific mapping files allow you to: - Override core mappings for specific companies - Add industry-specific concepts (automotive, technology, energy) - Handle company extension taxonomies - Maintain separation of concerns (one file per company) ### File Structure: {ticker}\_mappings.json **Current Location** (hardcoded): `edgar/xbrl/standardization/company_mappings/{ticker}_mappings.json` **Important Note**: Currently uses ticker as identifier, but **CIK-based identification is coming in v4.30.0/v4.31.0** to handle multi-ticker companies (GOOG/GOOGL, HEI.A/HEI.B). ### Complete Company Mapping Schema `{ "metadata": { "entity_identifier": "ticker_symbol", "company_name": "Full Company Name", "cik": "1234567", "priority": "high|medium|low", "created_date": "YYYY-MM-DD", "last_updated": "YYYY-MM-DD", "description": "Brief description of custom taxonomy needs" }, "concept_mappings": { "Standard Concept Label": [ "company:CustomTag", "company:AnotherCustomTag" ] }, "hierarchy_rules": { "Parent Concept": { "children": [ "Child Concept 1", "Child Concept 2" ], "description": "Optional explanation" } }, "business_context": { "primary_revenue_streams": ["stream1", "stream2"], "revenue_model": "product_and_service|subscription|manufacturing", "key_metrics": ["metric1", "metric2"], "industry": "industry_classification" } }` ### Real Example: Tesla (tsla\_mappings.json) `{ "metadata": { "entity_identifier": "tsla", "company_name": "Tesla, Inc.", "cik": "1318605", "priority": "high", "created_date": "2024-06-25", "last_updated": "2024-06-25", "description": "Tesla-specific concept mappings to handle automotive, energy, and service revenue streams" }, "concept_mappings": { "Automotive Revenue": [ "tsla:AutomotiveRevenue", "tsla:AutomotiveSales", "tsla:VehicleRevenue" ], "Automotive Leasing Revenue": [ "tsla:AutomotiveLeasing", "tsla:AutomotiveLeasingRevenue", "tsla:VehicleLeasingRevenue" ], "Energy Revenue": [ "tsla:EnergyGenerationAndStorageRevenue", "tsla:EnergyRevenue", "tsla:SolarRevenue", "tsla:EnergyStorageRevenue" ], "Service Revenue": [ "tsla:ServicesAndOtherRevenue", "tsla:ServiceRevenue", "tsla:SuperchargerRevenue" ] }, "hierarchy_rules": { "Revenue": { "children": [ "Automotive Revenue", "Energy Revenue", "Service Revenue" ] }, "Automotive Revenue": { "children": [ "Automotive Leasing Revenue" ] } }, "business_context": { "primary_revenue_streams": ["automotive", "energy", "services"], "revenue_model": "product_and_service", "key_metrics": ["vehicle_deliveries", "energy_deployments"], "industry": "automotive_technology" } }` ### Real Example: Microsoft (msft\_mappings.json) `{ "entity_info": { "name": "Microsoft Corporation", "cik": "0000789019", "ticker": "MSFT", "description": "Microsoft-specific concept mappings for unique business terminology" }, "concept_mappings": { "_comment_msft_revenue": "Microsoft uses specific revenue categorization that differs from standard tech companies", "Product Revenue": [ "msft:ProductRevenue", "msft:WindowsCommercialRevenue", "msft:WindowsConsumerRevenue", "msft:OfficeCommercialRevenue" ], "Service Revenue": [ "msft:ServiceRevenue", "msft:CloudServicesRevenue", "msft:ConsultingServicesRevenue" ], "Subscription Revenue": [ "msft:Office365CommercialRevenue", "msft:Office365ConsumerRevenue", "msft:DynamicsRevenue" ], "Platform Revenue": [ "msft:AzureRevenue", "msft:XboxContentAndServicesRevenue" ], "_comment_msft_expenses": "Microsoft has unique expense categorizations", "Sales and Marketing Expense": [ "msft:SalesAndMarketingExpense", "msft:AdvertisingAndPromotionExpense" ], "Technical Support Expense": [ "msft:TechnicalSupportExpense", "msft:CustomerSupportExpense" ] }, "hierarchy_rules": { "_comment": "Rules for handling Microsoft-specific hierarchical relationships", "revenue_hierarchy": { "parent": "Revenue", "children": ["Product Revenue", "Service Revenue", "Subscription Revenue", "Platform Revenue"], "calculation_rule": "sum" }, "expense_hierarchy": { "parent": "Operating Expenses", "children": ["Sales and Marketing Expense", "Technical Support Expense"], "calculation_rule": "sum" } } }` ### Creating Your Own Company Mapping **Step 1: Identify Company Extension Tags** `from edgar import Company company = Company("AAPL") filing = company.get_filings(form="10-K").latest() xbrl = filing.xbrl() # Find company-specific tags facts = xbrl.facts.query().to_dataframe() company_tags = facts[facts['concept'].str.startswith('aapl:')]['concept'].unique() print(f"Found {len(company_tags)} Apple-specific tags")` **Step 2: Create Mapping File** `{ "metadata": { "entity_identifier": "aapl", "company_name": "Apple Inc.", "cik": "0000320193", "priority": "high", "created_date": "2025-11-19", "last_updated": "2025-11-19", "description": "Apple-specific mappings for product categories" }, "concept_mappings": { "iPhone Revenue": [ "aapl:IPhoneRevenue", "aapl:IPhoneSales" ], "Services Revenue": [ "aapl:ServicesRevenue", "aapl:AppleCareRevenue", "aapl:ICloudRevenue" ] } }` **Step 3: Place File in Correct Location** `edgar/xbrl/standardization/company_mappings/aapl_mappings.json` **Current Limitation**: Must be inside the package directory (see Section 6). * * * 5\. Priority System and Ambiguous Tag Resolution ------------------------------------------------ ### The Ambiguous Tag Problem Over **200 XBRL tags** are inherently ambiguous and can map to multiple standard concepts depending on context. These fall into several categories: #### Category 1: Asset/Liability Ambiguity (12 tags) Tags that could be either assets or liabilities: `DeferredTaxAssetsLiabilitiesNet DeferredTaxAssetsLiabilitiesNetCurrent DeferredTaxAssetsLiabilitiesNetNoncurrent DerivativeAssetsLiabilitiesAtFairValueNet DeferredFinanceCostsCurrentNet DeferredFinanceCostsNoncurrentNet CustomerAdvancesAndProgressPaymentsForLongTermContractsOrPrograms DeferredTaxLiabilitiesGoodwillAndIntangibleAssets DeferredTaxLiabilitiesInvestments UnamortizedDebtIssuanceExpense DerivativeLiabilityFairValueGrossAsset` **Example: DeferredTaxAssetsLiabilitiesNet** This tag represents the NET of deferred tax assets and liabilities: - If positive → Deferred Tax Asset - If negative → Deferred Tax Liability **Resolution Strategy**: Use statement context and sign convention. #### Category 2: Current/Noncurrent Ambiguity (180+ tags) Tags that don't specify classification: `AccountsPayableCurrentAndNoncurrent AccountsPayableAndAccruedLiabilitiesCurrentAndNoncurrent AccountsReceivableGross AccountsReceivableNet DeferredRevenue ContractWithCustomerLiability ConvertibleDebt DebtInstrumentCarryingAmount` **Example: AccountsPayableCurrentAndNoncurrent** Without context, you can't determine if this should map to: - "Accounts Payable, Current" (on current liabilities) - "Accounts Payable, Noncurrent" (on long-term liabilities) - "Accounts Payable, Total" (parent concept) **Resolution Strategy**: Use calculation tree parent relationships and statement location. #### Category 3: Triple Ambiguity (1 tag) `DerivativeLiabilityFairValueGrossAsset` This tag is ambiguous in THREE dimensions: 1. Asset vs. Liability 2. Current vs. Noncurrent 3. Gross vs. Net **Resolution**: Requires comprehensive context analysis. #### Category 4: Total vs. Line Item Ambiguity `LiabilitiesNoncurrent` Some companies use this as: - **Total line**: Sum of all noncurrent liabilities - **Other line item**: "Other Noncurrent Liabilities" **Resolution**: Check if it has children in calculation tree or appears multiple times. ### Priority Levels Explained #### Priority 1: Core Mappings **Source**: `concept_mappings.json` **Use**: Base US-GAAP concepts that apply to all companies `# P1 mapping example "Revenue": [ "us-gaap:Revenue", "us-gaap:Revenues", "us-gaap:SalesRevenueNet" ]` **When Applied**: When no company-specific mapping exists. #### Priority 2: Company Mappings **Source**: `company_mappings/{ticker}_mappings.json` **Use**: Company-specific overrides and extensions `# P2 mapping example (Tesla) "Automotive Revenue": [ "tsla:AutomotiveRevenue", "tsla:AutomotiveSales" ]` **When Applied**: When company mapping file exists for the ticker. #### Priority 4: Detected Entity Match **Source**: Automatic detection from concept prefix **Use**: Boost priority when concept prefix matches company `# P4 boost example concept = "tsla:AutomotiveRevenue" # System detects "tsla:" prefix # Checks if "tsla" company mappings exist # Boosts priority from P2 → P4 for Tesla mappings` **When Applied**: When concept prefix matches a known company identifier. ### Context-Based Resolution The system uses multiple context signals to resolve ambiguous tags: #### 1\. Statement Type Context `context = { 'statement_type': 'BalanceSheet' # or 'IncomeStatement', 'CashFlowStatement' }` **Example Resolution**: `Tag: "DeferredTaxAssetsLiabilitiesNet" Statement: BalanceSheet Parent: "Assets" → Maps to: "Deferred Tax Assets" Tag: "DeferredTaxAssetsLiabilitiesNet" Statement: BalanceSheet Parent: "Liabilities" → Maps to: "Deferred Tax Liabilities"` #### 2\. Calculation Tree Relationships `context = { 'calculation_parent': 'us-gaap:AssetsCurrent', 'level': 1 }` **Example Resolution**: `Tag: "AccountsPayableCurrentAndNoncurrent" Parent: "AssetsCurrent" (impossible - payables are liabilities) → Check if sign is negative → Maps to: "Accounts Payable, Current" (negative in assets = liability) Tag: "AccountsPayableCurrentAndNoncurrent" Parent: "LiabilitiesCurrent" → Maps to: "Accounts Payable, Current" Tag: "AccountsPayableCurrentAndNoncurrent" Parent: "LiabilitiesNoncurrent" → Maps to: "Accounts Payable, Noncurrent"` #### 3\. Sign Conventions `# Positive value in assets section → Asset # Negative value in assets section → Liability (unusual presentation) # Check fact value and location` #### 4\. Position and Level `context = { 'position': 0, # First item in section 'level': 0, # Top level (total) 'is_total': True }` **Example Resolution**: `Tag: "LiabilitiesNoncurrent" Level: 0 Has children: Yes → Maps to: "Total Noncurrent Liabilities" Tag: "LiabilitiesNoncurrent" Level: 1 Has children: No → Maps to: "Other Noncurrent Liabilities"` ### Complete Ambiguous Tag List For reference, here are all identified ambiguous tags (from user @mpreiss9's analysis): **Asset/Liability Ambiguity (12 tags)**: `CustomerAdvancesAndProgressPaymentsForLongTermContractsOrPrograms DeferredFinanceCostsCurrentNet DeferredFinanceCostsNoncurrentNet DeferredTaxAssetsLiabilitiesNet DeferredTaxAssetsLiabilitiesNetCurrent DeferredTaxAssetsLiabilitiesNetNoncurrent DeferredTaxLiabilitiesGoodwillAndIntangibleAssets DeferredTaxLiabilitiesGoodwillAndIntangibleAssetsIntangibleAssets DeferredTaxLiabilitiesInvestments DerivativeAssetsLiabilitiesAtFairValueNet UnamortizedDebtIssuanceExpense DerivativeLiabilityFairValueGrossAsset` **Current/Noncurrent Ambiguity (180+ tags)** - Excerpt: `AccountsPayableAndAccruedLiabilitiesCurrentAndNoncurrent AccountsPayableAndOtherAccruedLiabilities AccountsPayableCurrentAndNoncurrent AccountsPayableOtherCurrentAndNoncurrent AccountsPayableTradeCurrentAndNoncurrent AccountsReceivableGross AccountsReceivableNet AccountsReceivableRelatedParties AccrualForTaxesOtherThanIncomeTaxesCurrentAndNoncurrent AccruedAdvertisingCurrentAndNoncurrent AccruedBonusesCurrentAndNoncurrent AccruedEmployeeBenefitsCurrentAndNoncurrent AccruedIncomeTaxes AccruedLiabilitiesCurrentAndNoncurrent AvailableForSaleSecuritiesDebtSecurities BusinessCombinationContingentConsiderationAsset BusinessCombinationContingentConsiderationLiability CapitalizedContractCostNet CapitalLeaseObligations ContractWithCustomerAssetNet ContractWithCustomerLiability ConvertibleDebt DebtInstrumentCarryingAmount DeferredRevenue DeferredTaxAssetsGross DeferredTaxAssetsNet DeferredTaxLiabilities DeferredTaxLiabilitiesNet DerivativeAssets DerivativeLiabilities EquitySecuritiesFvNi HeldToMaturitySecurities Investments LineOfCredit MarketableSecurities NotesAndLoansPayable OperatingLeaseLiability OtherAssets OtherLiabilities RestrictedCash` **See issue #494 comment for complete list of 200+ tags**. ### Implementing Custom Ambiguity Resolution If you need to handle ambiguous tags for your specific use case: **Option 1: Company-Specific Mapping** `{ "concept_mappings": { "Deferred Tax Assets": [ "us-gaap:DeferredTaxAssetsLiabilitiesNet" ] }, "notes": { "DeferredTaxAssetsLiabilitiesNet": "Company X always reports net deferred tax assets; never reports net liability" } }` **Option 2: Context Validation (Future Enhancement)** This is planned for v4.30.0: `from edgar.xbrl.standardization import MappingStore store = MappingStore() # Custom resolution function def custom_resolver(concept, context, value): if concept == "us-gaap:DeferredTaxAssetsLiabilitiesNet": if value > 0: return "Deferred Tax Assets" else: return "Deferred Tax Liabilities" return None store.add_custom_resolver(custom_resolver)` * * * 6\. Current Limitations ----------------------- This section documents **known limitations** of the current implementation and provides workarounds. These are on the roadmap for future releases. ### Limitation 1: Hardcoded Paths **Problem**: Mapping files MUST be inside the package directory. **Current Paths**: `edgar/xbrl/standardization/concept_mappings.json edgar/xbrl/standardization/company_mappings/{ticker}_mappings.json` **Why This is a Problem**: - Users can't maintain mappings outside the package - Difficult to version control custom mappings separately - Package updates overwrite custom mappings - Not suitable for production deployment workflows **Current Workaround**: Copy your custom mapping files into the package directory after installation: `# Find package location python -c "import edgar; print(edgar.__file__)" # Output: /path/to/site-packages/edgar/__init__.py # Copy your custom mappings cp my_custom_mappings.json /path/to/site-packages/edgar/xbrl/standardization/ cp company_mappings/* /path/to/site-packages/edgar/xbrl/standardization/company_mappings/` **Better Workaround** (Python script): `import shutil from pathlib import Path import edgar # Find package standardization directory edgar_path = Path(edgar.__file__).parent std_path = edgar_path / "xbrl" / "standardization" # Copy custom core mappings shutil.copy( "my_custom_mappings.json", std_path / "concept_mappings.json" ) # Copy company mappings company_dir = std_path / "company_mappings" for mapping_file in Path("my_company_mappings").glob("*_mappings.json"): shutil.copy(mapping_file, company_dir / mapping_file.name) print("Custom mappings installed successfully")` **Risks**: - Mappings are lost on package upgrade - Must re-run after each `pip install --upgrade edgartools` **Future Enhancement** (v4.30.0): `# Coming in v4.30.0 import os os.environ['EDGAR_MAPPINGS_PATH'] = '/path/to/my/mappings' # Or via constructor from edgar.xbrl.standardization import MappingStore store = MappingStore( core_path='/path/to/concept_mappings.json', company_dir='/path/to/company_mappings/' )` ### Limitation 2: Ticker-Based Identification **Problem**: Company mappings use ticker as identifier, not CIK. **Current Behavior**: `# File: company_mappings/msft_mappings.json { "metadata": { "entity_identifier": "msft", # Uses ticker "cik": "0000789019" # CIK is just metadata } }` **Why This is a Problem**: 1. **Multiple tickers per CIK**: 2. Alphabet: GOOG and GOOGL → Same CIK (1652044) 3. HEICO: HEI.A and HEI.B → Same CIK (46619) 4. **Ticker changes over time**: 5. Facebook → Meta (FB → META) 6. Google → Alphabet (GOOG → GOOGL split) 7. **CIK is the stable identifier** in SEC filings: 8. Every filing contains CIK 9. Ticker can be ambiguous or absent **Current Workaround**: Use the **primary ticker** and document alternatives in metadata: `{ "metadata": { "entity_identifier": "goog", "company_name": "Alphabet Inc.", "cik": "0001652044", "alternative_tickers": ["GOOGL", "GOOG"], "notes": "Use GOOG as primary; applies to both Class A (GOOGL) and Class C (GOOG)" } }` Then create a symlink or duplicate file: `cd company_mappings/ cp goog_mappings.json googl_mappings.json` **Future Enhancement** (v4.30.0 - v4.31.0): `{ "metadata": { "entity_identifier": "0001652044", # CIK is primary identifier "tickers": ["GOOG", "GOOGL"], # Multiple tickers supported "primary_ticker": "GOOG" } }` **Timeline**: - **v4.30.0**: Add CIK-based lookup support (dual lookup during transition) - **v4.31.0**: Default to CIK-based identification - **v5.0.0**: Deprecate ticker-based identification ### Limitation 3: JSON-Only Format **Problem**: No native CSV support for mapping files. **Why This is a Problem**: - CSV is easier to edit in Excel or Google Sheets - Easier to detect duplicates with spreadsheet tools - Simpler to sort, filter, and validate mappings - More accessible for non-technical users **Current Workaround**: See Section 8 (CSV Workflow) for utilities. **Quick CSV-to-JSON Converter**: `import csv import json from collections import defaultdict def csv_to_mappings(csv_path, json_path): """Convert CSV mapping file to JSON format.""" mappings = defaultdict(list) with open(csv_path, 'r') as f: reader = csv.DictReader(f) for row in reader: standard_concept = row['standard_concept'] company_concept = row['company_concept'] mappings[standard_concept].append(company_concept) with open(json_path, 'w') as f: json.dump(dict(mappings), f, indent=2) print(f"Converted {len(mappings)} concepts") # Usage csv_to_mappings('my_mappings.csv', 'concept_mappings.json')` **Expected CSV Format**: `standard_concept,company_concept,notes Revenue,us-gaap:Revenue,Standard revenue tag Revenue,us-gaap:Revenues,Alternative spelling Automotive Revenue,tsla:AutomotiveRevenue,Tesla-specific` **Future Enhancement** (v4.30.0): `# Native CSV support - auto-detect from extension from edgar.xbrl.standardization import MappingStore # Automatically loads CSV or JSON based on extension store = MappingStore(source="my_mappings.csv")` * * * 7\. Validation Techniques ------------------------- Validation is critical when working with custom mappings across 200+ companies. Here are proven techniques for ensuring mapping quality. ### The Balance Sheet Validation Principle The fundamental validation for balance sheets: `# Core accounting equation Assets = Liabilities + Equity # Detailed validation Total Assets = Current Assets + Noncurrent Assets Total Assets = Sum(all individual asset line items) Total Liabilities = Current Liabilities + Noncurrent Liabilities Total Equity = Common Stock + Retained Earnings + Other Equity Assets = Liabilities + Equity` ### Balance Sheet Validation Code `def validate_balance_sheet(xbrl, period_key): """Validate that balance sheet balances using mapped concepts.""" facts = xbrl.facts.query().by_period_key(period_key).to_dataframe() # Get key totals total_assets = facts[facts['label'] == 'Total Assets']['value'].sum() current_assets = facts[facts['label'] == 'Total Current Assets']['value'].sum() noncurrent_assets = facts[facts['label'] == 'Total Non Current Assets']['value'].sum() total_liabilities = facts[facts['label'] == 'Total Liabilities']['value'].sum() total_equity = facts[facts['label'] == "Total Stockholders' Equity"]['value'].sum() # Validation 1: Assets = Current + Noncurrent if noncurrent_assets: # Some companies don't report noncurrent separately assets_check = abs(total_assets - (current_assets + noncurrent_assets)) < 1.0 if not assets_check: print(f"WARNING: Assets don't balance: {total_assets} != {current_assets} + {noncurrent_assets}") # Validation 2: Assets = Liabilities + Equity accounting_equation = abs(total_assets - (total_liabilities + total_equity)) < 1.0 if not accounting_equation: print(f"ERROR: Accounting equation violated: {total_assets} != {total_liabilities} + {total_equity}") return False # Validation 3: Sum of line items = Total Assets asset_line_items = facts[ (facts['concept'].str.contains('Asset')) & (facts['label'] != 'Total Assets') ]['value'].sum() detail_check = abs(total_assets - asset_line_items) < 1.0 if not detail_check: print(f"WARNING: Asset line items don't sum to total: {total_assets} != {asset_line_items}") return accounting_equation # Usage filing = Company("AAPL").get_filings(form="10-K").latest() xbrl = filing.xbrl() period_key = xbrl.reporting_periods[0]['key'] is_valid = validate_balance_sheet(xbrl, period_key) print(f"Balance sheet valid: {is_valid}")` ### Income Statement Validation Income statement validation is **more complex** due to: - Variable presentation formats - Sign convention inconsistencies (some expenses are positive, some negative) - Different levels of detail across companies `def validate_income_statement(xbrl, period_key): """Validate income statement using anchored approach.""" facts = xbrl.facts.query().by_period_key(period_key).to_dataframe() # Anchor points (always present and unambiguous) revenue = facts[facts['label'] == 'Revenue']['value'].sum() net_income = facts[facts['label'] == 'Net Income']['value'].sum() if revenue == 0 or net_income == 0: print("ERROR: Missing anchor points (Revenue or Net Income)") return False # Get all expense items (with sign normalization) expense_concepts = [ 'Cost of Revenue', 'Research and Development Expense', 'Selling, General and Administrative Expense', 'Interest Expense', 'Income Tax Expense' ] total_expenses = 0 for concept in expense_concepts: expense_facts = facts[facts['label'] == concept]['value'] if len(expense_facts) > 0: expense_value = expense_facts.sum() # Normalize to positive (expenses reduce income) if expense_value < 0: expense_value = abs(expense_value) total_expenses += expense_value # Check if Revenue - Expenses ≈ Net Income # Allow for other income/expense not captured calculated_ni = revenue - total_expenses difference = abs(calculated_ni - net_income) # Difference should be small (other income/expense) acceptable_diff = abs(revenue) * 0.1 # 10% tolerance for other items if difference > acceptable_diff: print(f"WARNING: Income statement doesn't reconcile:") print(f" Revenue: {revenue:,.0f}") print(f" Total Expenses: {total_expenses:,.0f}") print(f" Calculated NI: {calculated_ni:,.0f}") print(f" Reported NI: {net_income:,.0f}") print(f" Difference: {difference:,.0f} (acceptable: {acceptable_diff:,.0f})") return False return True` ### Unmapped Tag Detection Detect XBRL tags that aren't mapped to standard concepts: `def find_unmapped_tags(xbrl, mapper): """Find all XBRL tags that don't map to standard concepts.""" unmapped = [] # Get all unique concepts facts = xbrl.facts.query().to_dataframe() concepts = facts['concept'].unique() for concept in concepts: # Try to map each concept label = facts[facts['concept'] == concept]['label'].iloc[0] context = {'statement_type': 'Unknown'} standard_concept = mapper.map_concept(concept, label, context) if standard_concept is None: unmapped.append({ 'concept': concept, 'label': label, 'occurrences': len(facts[facts['concept'] == concept]) }) # Sort by occurrences (most common first) unmapped.sort(key=lambda x: x['occurrences'], reverse=True) return unmapped # Usage from edgar.xbrl.standardization import MappingStore, ConceptMapper store = MappingStore() mapper = ConceptMapper(store) unmapped = find_unmapped_tags(xbrl, mapper) print(f"Found {len(unmapped)} unmapped tags") for tag in unmapped[:10]: # Top 10 print(f" {tag['concept']}: '{tag['label']}' ({tag['occurrences']} occurrences)")` ### Logging Unmapped Tags for Review Create a log file of unmapped tags with suggested mappings: `import csv from difflib import get_close_matches def log_unmapped_tags(xbrl, mapper, output_path='unmapped_tags.csv'): """Create CSV log of unmapped tags with suggested standard concepts.""" unmapped = find_unmapped_tags(xbrl, mapper) # Get all standard concepts for matching standard_concepts = list(store.mappings.keys()) with open(output_path, 'w', newline='') as f: writer = csv.writer(f) writer.writerow([ 'company_concept', 'label', 'occurrences', 'suggested_mapping', 'confidence', 'cik', 'notes' ]) for tag in unmapped: # Find closest matching standard concept matches = get_close_matches( tag['label'], standard_concepts, n=1, cutoff=0.6 ) suggested = matches[0] if matches else "MANUAL_REVIEW_NEEDED" confidence = "high" if matches and len(matches[0]) else "low" writer.writerow([ tag['concept'], tag['label'], tag['occurrences'], suggested, confidence, xbrl.entity_identifier, "" # Manual notes column ]) print(f"Wrote {len(unmapped)} unmapped tags to {output_path}") print("Review file and add to concept_mappings.json") # Usage - process all companies companies = ["AAPL", "MSFT", "GOOGL", "TSLA"] for ticker in companies: company = Company(ticker) filing = company.get_filings(form="10-K").latest() xbrl = filing.xbrl() log_unmapped_tags(xbrl, mapper, f"unmapped_{ticker}.csv")` ### Validation Utility Script Comprehensive validation script for batch processing: `def validate_company_mappings(ticker, form="10-K", years=3): """Validate mappings for a company across multiple years.""" company = Company(ticker) filings = company.get_filings(form=form).head(years) results = [] for filing in filings: print(f"\nValidating {ticker} {filing.filing_date}...") try: xbrl = filing.xbrl() period_key = xbrl.reporting_periods[0]['key'] # Run validations bs_valid = validate_balance_sheet(xbrl, period_key) is_valid = validate_income_statement(xbrl, period_key) unmapped = find_unmapped_tags(xbrl, mapper) result = { 'ticker': ticker, 'filing_date': filing.filing_date, 'balance_sheet_valid': bs_valid, 'income_statement_valid': is_valid, 'unmapped_count': len(unmapped), 'total_concepts': len(xbrl.facts.query().to_dataframe()['concept'].unique()) } results.append(result) print(f" Balance Sheet: {'✓' if bs_valid else '✗'}") print(f" Income Statement: {'✓' if is_valid else '✗'}") print(f" Unmapped: {len(unmapped)}") except Exception as e: print(f" ERROR: {e}") results.append({ 'ticker': ticker, 'filing_date': filing.filing_date, 'error': str(e) }) return results # Batch validation companies = ["AAPL", "MSFT", "GOOGL", "TSLA", "AMZN"] all_results = [] for ticker in companies: results = validate_company_mappings(ticker) all_results.extend(results) # Summary valid_count = sum(1 for r in all_results if r.get('balance_sheet_valid', False)) print(f"\nOverall: {valid_count}/{len(all_results)} filings validated successfully")` * * * 8\. CSV Workflow ---------------- While EdgarTools currently uses JSON for mappings, many users prefer CSV for editing. This section provides utilities for CSV-based workflows. **Note**: Native CSV support is planned for v4.29.0/v4.30.0. ### Why CSV for Mapping Management? **Advantages**: - **Excel editing**: Use familiar spreadsheet tools - **Duplicate detection**: Sort columns to find duplicates easily - **Filtering**: Quick filtering by standard concept or company - **Validation**: Formulas can check for consistency - **Collaboration**: Easier for non-technical team members ### CSV Format Specification **Standard Format**: `standard_concept,company_concept,company_cik,priority,notes Revenue,us-gaap:Revenue,,1,Core GAAP concept Revenue,us-gaap:Revenues,,1,Alternative spelling Automotive Revenue,tsla:AutomotiveRevenue,1318605,2,Tesla-specific Automotive Revenue,tsla:VehicleRevenue,1318605,2,Alternative Tesla tag` **Columns**: - `standard_concept`: The standardized label (e.g., "Revenue") - `company_concept`: The XBRL tag (e.g., "us-gaap:Revenue") - `company_cik`: Optional CIK for company-specific mappings (empty for core) - `priority`: 1=core, 2=company-specific (optional, for reference) - `notes`: Explanation, context, or validation notes ### Export Mappings to CSV `import csv from edgar.xbrl.standardization import MappingStore def export_mappings_to_csv(store: MappingStore, output_path: str): """Export MappingStore to CSV format for editing.""" rows = [] # Export core mappings (priority 1) for standard_concept, company_concepts in store.mappings.items(): for company_concept in company_concepts: rows.append({ 'standard_concept': standard_concept, 'company_concept': company_concept, 'company_cik': '', 'priority': 1, 'notes': 'Core mapping' }) # Export company-specific mappings (priority 2) for entity_id, company_data in store.company_mappings.items(): cik = company_data.get('metadata', {}).get('cik', '') concept_mappings = company_data.get('concept_mappings', {}) for standard_concept, company_concepts in concept_mappings.items(): for company_concept in company_concepts: rows.append({ 'standard_concept': standard_concept, 'company_concept': company_concept, 'company_cik': cik, 'priority': 2, 'notes': f'Company-specific: {entity_id}' }) # Write to CSV with open(output_path, 'w', newline='') as f: fieldnames = ['standard_concept', 'company_concept', 'company_cik', 'priority', 'notes'] writer = csv.DictWriter(f, fieldnames=fieldnames) writer.writeheader() writer.writerows(rows) print(f"Exported {len(rows)} mappings to {output_path}") # Usage store = MappingStore() export_mappings_to_csv(store, 'all_mappings.csv')` ### Import Mappings from CSV `import csv from collections import defaultdict import json def import_mappings_from_csv(csv_path: str): """Import mappings from CSV and generate JSON files.""" core_mappings = defaultdict(list) company_mappings = defaultdict(lambda: defaultdict(list)) with open(csv_path, 'r') as f: reader = csv.DictReader(f) for row in reader: standard_concept = row['standard_concept'] company_concept = row['company_concept'] cik = row.get('company_cik', '').strip() if cik: # Company-specific mapping company_mappings[cik][standard_concept].append(company_concept) else: # Core mapping core_mappings[standard_concept].append(company_concept) # Save core mappings with open('concept_mappings.json', 'w') as f: json.dump(dict(core_mappings), f, indent=2) print(f"Saved core mappings: {len(core_mappings)} concepts") # Save company-specific mappings for cik, mappings in company_mappings.items(): # Find ticker from CIK (simplified - you'd need a CIK-to-ticker lookup) ticker = f"cik{cik}" # Placeholder company_data = { "metadata": { "entity_identifier": ticker, "cik": cik, "priority": "high", "created_date": "2025-11-19" }, "concept_mappings": dict(mappings) } filename = f"{ticker}_mappings.json" with open(filename, 'w') as f: json.dump(company_data, f, indent=2) print(f"Saved company mappings: {filename}") # Usage import_mappings_from_csv('all_mappings.csv')` ### Excel Editing Workflow **Step 1: Export to CSV** `from edgar.xbrl.standardization import MappingStore store = MappingStore() export_mappings_to_csv(store, 'edgartools_mappings.csv')` **Step 2: Open in Excel** - Open `edgartools_mappings.csv` in Excel or Google Sheets - Use Excel features: - **Sort** by `standard_concept` to group related mappings - **Filter** by `company_cik` to see company-specific mappings - **Conditional Formatting** to highlight duplicates - **Find & Replace** for bulk updates **Step 3: Duplicate Detection in Excel** Formula in column F (next to your data): `=COUNTIFS($B:$B,B2,$A:$A,A2)>1` This highlights if the same `company_concept` maps to the same `standard_concept` multiple times. **Step 4: Validation in Excel** Add a validation column with this formula: `=IF(ISBLANK(B2), "Missing concept", IF(ISBLANK(A2), "Missing label", IF(AND(C2<>"", NOT(ISNUMBER(C2))), "Invalid CIK", "OK")))` **Step 5: Import Back to JSON** `import_mappings_from_csv('edgartools_mappings.csv')` ### Single File vs Multiple Files Two approaches for managing 200+ companies: #### Approach 1: Single CSV File (Recommended for Excel Users) **Structure**: `standard_concept,company_concept,company_cik,ticker,notes Revenue,us-gaap:Revenue,,,Core GAAP Automotive Revenue,tsla:AutomotiveRevenue,1318605,TSLA,Tesla-specific Energy Revenue,tsla:EnergyRevenue,1318605,TSLA,Tesla energy Product Revenue,msft:ProductRevenue,789019,MSFT,Microsoft` **Advantages**: - Easy to search across all companies - Single source of truth - Easy duplicate detection - Better for bulk operations **Disadvantages**: - Large file size (200 companies = 10,000+ rows) - Merge conflicts in version control - Slower to load #### Approach 2: Multiple JSON Files (Current EdgarTools Approach) **Structure**: `company_mappings/ aapl_mappings.json msft_mappings.json tsla_mappings.json googl_mappings.json ...` **Advantages**: - Modular (edit one company at a time) - Better for version control (fewer merge conflicts) - Faster loading (only load relevant companies) - Clear ownership (one file per company) **Disadvantages**: - Harder to find duplicates across companies - More files to manage - Need tooling to search across all files #### Hybrid Approach (Best of Both Worlds) Use CSV as master source, generate JSON files: `def csv_to_company_json_files(csv_path: str, output_dir: str): """Convert single CSV to multiple company JSON files.""" import csv import json from pathlib import Path from collections import defaultdict Path(output_dir).mkdir(exist_ok=True) # Group by CIK company_data = defaultdict(lambda: { 'metadata': {}, 'concept_mappings': defaultdict(list) }) with open(csv_path, 'r') as f: reader = csv.DictReader(f) for row in reader: cik = row.get('company_cik', '').strip() if not cik: continue # Skip core mappings ticker = row.get('ticker', f'cik{cik}').lower() # Set metadata if not company_data[ticker]['metadata']: company_data[ticker]['metadata'] = { 'entity_identifier': ticker, 'cik': cik, 'priority': 'high' } # Add mapping standard = row['standard_concept'] concept = row['company_concept'] company_data[ticker]['concept_mappings'][standard].append(concept) # Write files for ticker, data in company_data.items(): # Convert defaultdict to regular dict data['concept_mappings'] = dict(data['concept_mappings']) filename = Path(output_dir) / f"{ticker}_mappings.json" with open(filename, 'w') as f: json.dump(data, f, indent=2) concept_count = len(data['concept_mappings']) print(f"Created {filename} with {concept_count} concepts") # Usage csv_to_company_json_files( 'master_mappings.csv', 'company_mappings/' )` **Recommended Workflow for 200+ Companies**: 1. Maintain master CSV file: `edgartools_master_mappings.csv` 2. Edit in Excel (easy duplicate detection, filtering) 3. Run conversion script to generate JSON files 4. Deploy JSON files to package directory 5. Version control both CSV (master) and JSON (generated) * * * 9\. Real-World Examples ----------------------- This section explains existing company mapping files with detailed annotations. ### Example 1: Tesla (Automotive + Energy) Tesla has a complex revenue structure combining automotive sales, leasing, and energy generation/storage. **File**: `company_mappings/tsla_mappings.json` `{ "metadata": { "entity_identifier": "tsla", "company_name": "Tesla, Inc.", "cik": "1318605", "priority": "high", "created_date": "2024-06-25", "last_updated": "2024-06-25", "description": "Tesla-specific concept mappings to handle automotive, energy, and service revenue streams" }, "concept_mappings": { "Automotive Revenue": [ "tsla:AutomotiveRevenue", "tsla:AutomotiveSales", "tsla:VehicleRevenue" ], "Automotive Leasing Revenue": [ "tsla:AutomotiveLeasing", "tsla:AutomotiveLeasingRevenue", "tsla:VehicleLeasingRevenue" ], "Energy Revenue": [ "tsla:EnergyGenerationAndStorageRevenue", "tsla:EnergyRevenue", "tsla:SolarRevenue", "tsla:EnergyStorageRevenue" ], "Service Revenue": [ "tsla:ServicesAndOtherRevenue", "tsla:ServiceRevenue", "tsla:SuperchargerRevenue" ] }, "hierarchy_rules": { "Revenue": { "children": [ "Automotive Revenue", "Energy Revenue", "Service Revenue" ] }, "Automotive Revenue": { "children": [ "Automotive Leasing Revenue" ] } }, "business_context": { "primary_revenue_streams": ["automotive", "energy", "services"], "revenue_model": "product_and_service", "key_metrics": ["vehicle_deliveries", "energy_deployments"], "industry": "automotive_technology" } }` **Key Design Decisions**: 1. **Granular Revenue Breakdown**: 2. Separate automotive sales from leasing (different economics) 3. Distinguish energy from automotive (different growth drivers) 4. Services as distinct category (recurring revenue) 5. **Hierarchy Rules**: 6. `Revenue` is parent of three main streams 7. `Automotive Revenue` contains `Automotive Leasing Revenue` as child 8. This ensures proper nesting in financial statements 9. **Multiple Tag Variations**: 10. Tesla has changed tag names over time (`AutomotiveRevenue` vs `AutomotiveSales`) 11. All variations map to same standard concept for consistency **Usage Example**: `from edgar import Company tesla = Company("TSLA") filing = tesla.get_filings(form="10-K").latest() xbrl = filing.xbrl() # Get standardized income statement income = xbrl.statements.income_statement() # Tesla-specific revenue line items will appear as: # - Automotive Revenue (instead of generic "Revenue") # - Automotive Leasing Revenue # - Energy Revenue # - Service Revenue` ### Example 2: Microsoft (Technology Platform) Microsoft has platform-based revenue (Azure, Office 365, Dynamics) requiring specialized mapping. **File**: `company_mappings/msft_mappings.json` `{ "entity_info": { "name": "Microsoft Corporation", "cik": "0000789019", "ticker": "MSFT", "description": "Microsoft-specific concept mappings for unique business terminology" }, "concept_mappings": { "_comment_msft_revenue": "Microsoft uses specific revenue categorization that differs from standard tech companies", "Product Revenue": [ "msft:ProductRevenue", "msft:WindowsCommercialRevenue", "msft:WindowsConsumerRevenue", "msft:OfficeCommercialRevenue" ], "Service Revenue": [ "msft:ServiceRevenue", "msft:CloudServicesRevenue", "msft:ConsultingServicesRevenue" ], "Subscription Revenue": [ "msft:Office365CommercialRevenue", "msft:Office365ConsumerRevenue", "msft:DynamicsRevenue" ], "Platform Revenue": [ "msft:AzureRevenue", "msft:XboxContentAndServicesRevenue" ], "_comment_msft_expenses": "Microsoft has unique expense categorizations for sales and marketing vs G&A", "Sales and Marketing Expense": [ "msft:SalesAndMarketingExpense", "msft:AdvertisingAndPromotionExpense" ], "Technical Support Expense": [ "msft:TechnicalSupportExpense", "msft:CustomerSupportExpense" ] }, "hierarchy_rules": { "_comment": "Rules for handling Microsoft-specific hierarchical relationships", "revenue_hierarchy": { "parent": "Revenue", "children": ["Product Revenue", "Service Revenue", "Subscription Revenue", "Platform Revenue"], "calculation_rule": "sum" }, "expense_hierarchy": { "parent": "Operating Expenses", "children": ["Sales and Marketing Expense", "Technical Support Expense"], "calculation_rule": "sum" } } }` **Key Design Decisions**: 1. **Four Revenue Categories**: 2. **Product**: Traditional software sales (Windows, Office perpetual licenses) 3. **Service**: Consulting, support services 4. **Subscription**: Recurring revenue (Office 365, Dynamics) 5. **Platform**: Cloud platforms (Azure, Xbox services) 6. **Expense Granularity**: 7. Separates sales/marketing from technical support 8. Reflects Microsoft's investment in customer success teams 9. **Hierarchy Rules with Calculation**: 10. Explicit `calculation_rule: sum` indicates children should sum to parent 11. Validation can check this relationship **Usage Example**: `msft = Company("MSFT") filing = msft.get_filings(form="10-K").latest() xbrl = filing.xbrl() # Analyze revenue mix facts = xbrl.facts.query().by_statement_type("IncomeStatement").to_dataframe() revenue_breakdown = facts[facts['label'].str.contains('Revenue')][['label', 'value']] print(revenue_breakdown) # Output: # label value # Product Revenue 75,000,000,000 # Service Revenue 25,000,000,000 # Subscription Revenue 60,000,000,000 # Platform Revenue 40,000,000,000 # Revenue 200,000,000,000` ### Example 3: Berkshire Hathaway (Conglomerate) Berkshire Hathaway is a diversified holding company with insurance, utilities, railroads, and manufacturing. **File**: `company_mappings/brka_mappings.json` `{ "concept_mappings": { "Sales and Service Revenue": [ "brka:SalesAndServiceRevenue" ] }, "hierarchy_rules": { "Revenue": { "components": [ "Sales and Service Revenue", "Operating Lease Revenue" ], "description": "Total revenue comprises sales/service revenue and operating lease income for holding company" } }, "business_context": { "entity_type": "holding_company", "industry": "diversified_conglomerate", "description": "Berkshire Hathaway operates diverse businesses including insurance, utilities, railroads, and manufacturing" } }` **Key Design Decisions**: 1. **Minimal Customization**: 2. Berkshire uses mostly standard US-GAAP tags 3. Only needs mapping for unique revenue categorization 4. **Lease Revenue Separation**: 5. Operating lease revenue (equipment leasing subsidiaries) 6. Separated from core sales/service revenue 7. **Business Context**: 8. Documents the holding company structure 9. Helps interpreters understand diverse revenue sources **Why So Simple?**: - Berkshire's filings primarily use standard US-GAAP taxonomy - Conglomerates often don't need extensive custom tags - Industry-specific tags are used by individual subsidiaries (not parent) ### Example 4: Industrial Company Template For users managing 200+ industrial companies, here's a template: `{ "metadata": { "entity_identifier": "ticker", "company_name": "Company Name", "cik": "0000000000", "priority": "medium", "created_date": "2025-11-19", "last_updated": "2025-11-19", "description": "Industrial company with manufacturing operations", "industry": "industrial_manufacturing" }, "concept_mappings": { "_comment": "Common industrial company customizations", "Product Sales": [ "company:ProductSales", "company:ManufacturedGoodsSales" ], "Raw Materials Inventory": [ "company:RawMaterialsInventory" ], "Work in Process Inventory": [ "company:WorkInProcessInventory" ], "Finished Goods Inventory": [ "company:FinishedGoodsInventory" ], "Manufacturing Overhead": [ "company:ManufacturingOverhead", "company:FactoryOverhead" ] }, "hierarchy_rules": { "Inventory": { "children": [ "Raw Materials Inventory", "Work in Process Inventory", "Finished Goods Inventory" ], "calculation_rule": "sum", "description": "Manufacturing inventory breakdown" } }, "business_context": { "primary_revenue_streams": ["product_sales"], "revenue_model": "manufacturing", "key_metrics": ["inventory_turnover", "production_efficiency"], "industry": "industrial_manufacturing", "notes": "Focus on inventory management and cost of goods sold structure" } }` **Adaptation for Your Companies**: 1. Copy this template 2. Replace `company:` prefix with actual company prefix 3. Add industry-specific concepts (automotive parts, chemicals, etc.) 4. Customize inventory structure based on business model * * * 10\. Future Enhancements ------------------------ This section outlines the roadmap for standardization improvements based on user feedback. ### Version 4.30.0 (Next 1-2 Months) **Focus**: Configuration and CSV Support #### 1\. Configurable Mapping Paths **Problem Solved**: Users can maintain mappings outside package directory. **Implementation**: `# Environment variable configuration import os os.environ['EDGAR_CORE_MAPPINGS'] = '/path/to/my/concept_mappings.json' os.environ['EDGAR_COMPANY_MAPPINGS_DIR'] = '/path/to/my/company_mappings/' # Library loads from custom paths from edgar.xbrl.standardization import MappingStore store = MappingStore() # Automatically uses env var paths` **Alternative: Constructor parameters**: `store = MappingStore( core_mappings_path='/path/to/concept_mappings.json', company_mappings_dir='/path/to/company_mappings/' )` **Benefits**: - Separate version control for mappings - Mappings survive package upgrades - Multiple mapping sets for different use cases #### 2\. Native CSV Format Support **Problem Solved**: Excel-based workflows without conversion scripts. **Implementation**: `# Auto-detect format from extension store = MappingStore(core_mappings_path='my_mappings.csv') # Explicit format specification store = MappingStore( core_mappings_path='my_mappings.txt', format='csv' )` **CSV Format**: `standard_concept,company_concept,notes Revenue,us-gaap:Revenue,Core GAAP tag Revenue,us-gaap:Revenues,Alternative spelling` **Benefits**: - No conversion scripts needed - Direct Excel editing - Easier duplicate detection #### 3\. Enhanced Validation Tools **Problem Solved**: Automated mapping quality checks. **Implementation**: `from edgar.xbrl.standardization import MappingValidator validator = MappingValidator(store) # Validate balance sheet balancing report = validator.validate_company( ticker="AAPL", form="10-K", years=3 ) print(report.summary()) # Output: # ✓ Balance Sheet: 3/3 periods balanced # ✓ Income Statement: 3/3 periods validated # ⚠ Unmapped tags: 12 concepts need mapping` **Features**: - Batch validation across multiple companies - Balance sheet equation checking - Income statement reconciliation - Coverage reports (% of concepts mapped) ### Version 4.31.0 (2-3 Months) **Focus**: CIK-Based Identification #### 1\. CIK as Primary Identifier **Problem Solved**: Handle multi-ticker companies (GOOG/GOOGL, HEI.A/HEI.B). **Implementation**: `{ "metadata": { "entity_identifier": "0001652044", "cik": "0001652044", "tickers": ["GOOG", "GOOGL"], "primary_ticker": "GOOG", "company_name": "Alphabet Inc." } }` **File Naming**: `company_mappings/ cik0001652044_mappings.json # CIK-based naming # OR legacy support: goog_mappings.json # Ticker-based naming (still supported)` #### 2\. Dual Lookup Support **During Transition**: Support both ticker and CIK lookups. `# Both work store.get_company_mappings(ticker="GOOG") store.get_company_mappings(cik="0001652044")` #### 3\. Migration Tool **Help users migrate** from ticker-based to CIK-based files. `from edgar.xbrl.standardization import migrate_to_cik # Migrate all ticker-based files to CIK-based migrate_to_cik( input_dir='company_mappings/', output_dir='company_mappings_cik/', cik_lookup_file='ticker_to_cik.csv' )` ### Version 5.0.0 (Major Release) **Focus**: Advanced Features and ML Integration #### 1\. JSON-Loaded StandardConcept **Problem Solved**: StandardConcept enum becomes fully data-driven. **Current**: `# Enum is hardcoded in Python class StandardConcept(str, Enum): REVENUE = "Revenue" TOTAL_ASSETS = "Total Assets"` **Future**: `# Enum loaded from JSON at runtime StandardConcept = load_concepts_from_json('standard_concepts.json') # Users can extend without touching Python code` #### 2\. Concept Marketplace/Repository **Problem Solved**: Share mappings across community. **Vision**: `from edgar.xbrl.standardization import ConceptMarketplace marketplace = ConceptMarketplace() # Download community mappings marketplace.install('industrial-companies-pack') marketplace.install('tech-companies-pack') # Share your mappings marketplace.publish( 'my-custom-mappings', description='Custom mappings for 200+ industrial firms', companies=['AAPL', 'MSFT', ...], license='MIT' )` **Features**: - Community-contributed mappings - Rating and review system - Automatic updates - Industry-specific packs #### 3\. ML-Based Concept Inference **Problem Solved**: Automatically suggest mappings for unmapped tags. **Implementation**: `from edgar.xbrl.standardization import MLConceptMapper ml_mapper = MLConceptMapper() # Train on existing mappings ml_mapper.train(store.mappings) # Suggest mappings for unmapped concepts suggestion = ml_mapper.suggest( concept='company:CustomRevenueConcept', label='Sales of Manufactured Goods', context={'statement_type': 'IncomeStatement'} ) print(suggestion) # Output: # Suggested: "Product Revenue" # Confidence: 0.89 # Similar concepts: ["Revenue", "Product Sales", "Sales"]` **Features**: - Learn from existing mappings - Context-aware suggestions - Confidence scoring - Interactive review workflow #### 4\. Advanced Validation Framework **Problem Solved**: Comprehensive statement validation. `from edgar.xbrl.standardization import ValidationFramework framework = ValidationFramework(store) # Define custom validation rules @framework.rule(statement='BalanceSheet', severity='error') def validate_accounting_equation(facts): assets = facts.get('Total Assets') liabilities = facts.get('Total Liabilities') equity = facts.get("Total Stockholders' Equity") if abs(assets - (liabilities + equity)) > 1.0: return ValidationError("Accounting equation violated") return None # Run validation results = framework.validate_company('AAPL', years=10) results.generate_report('validation_report.html')` ### Timeline Summary | Feature | Version | Timeline | Status | | --- | --- | --- | --- | | Configurable paths | v4.30.0 | 1-2 months | Planned | | Native CSV support | v4.30.0 | 1-2 months | Planned | | Enhanced validation | v4.30.0 | 1-2 months | Planned | | CIK-based identification | v4.31.0 | 2-3 months | Planned | | Dual lookup support | v4.31.0 | 2-3 months | Planned | | Migration tool | v4.31.0 | 2-3 months | Planned | | JSON StandardConcept | v5.0.0 | 6-12 months | Under consideration | | Concept marketplace | v5.0.0 | 6-12 months | Under consideration | | ML concept inference | v5.0.0 | 6-12 months | Research phase | ### Providing Feedback Your feedback shapes these enhancements. To contribute: 1. **GitHub Issues**: Comment on issue #494 or create new issues 2. **Feature Requests**: Use the feature request template 3. **User Stories**: Share your specific use cases 4. **Beta Testing**: Volunteer to test pre-release versions **Contact**: - GitHub: https://github.com/dgunning/edgartools/issues/494 - Discussions: https://github.com/dgunning/edgartools/discussions * * * Summary and Quick Reference --------------------------- ### When to Customize Standardization ✅ **Yes, customize when**: - Managing 200+ companies with diverse taxonomies - Industry-specific valuations (industrial, automotive, tech) - Building models requiring consistent data structure - Statement balancing is critical to your workflow ❌ **No, use defaults when**: - Analyzing 1-10 companies - Standard US-GAAP concepts are sufficient - Quick analysis or exploration - Don't need custom taxonomy support ### Quick Decision Tree `Do you analyze 200+ companies? ├─ Yes → Use custom company-specific mappings (Section 4) │ └─ CSV workflow for easier management (Section 8) └─ No → Do you need industry-specific concepts? ├─ Yes → Use custom core mappings (Section 3) └─ No → Use default StandardConcept mappings` ### Essential Resources | Task | Section | Key File | | --- | --- | --- | | Understand architecture | Section 2 | `core.py` | | Add core mappings | Section 3 | `concept_mappings.json` | | Create company mappings | Section 4 | `{ticker}_mappings.json` | | Resolve ambiguous tags | Section 5 | Your context analysis | | Work around limitations | Section 6 | Installation scripts | | Validate mappings | Section 7 | Validation utilities | | Use CSV workflow | Section 8 | CSV utilities | | Learn from examples | Section 9 | Tesla, Microsoft files | ### Key Concepts Clarified | Concept | What It Is | What It's NOT | | --- | --- | --- | | **StandardConcept Enum** | IDE convenience, type safety | NOT the mapping data | | **JSON Mappings** | Source of truth for mappings | NOT just for reference | | **Priority System** | Conflict resolution | NOT just ordering | | **CIK** | Stable company identifier | NOT ticker (which changes) | | **Context** | Ambiguity resolution | NOT just metadata | ### Contact and Support * **GitHub Issue**: #494 * **Documentation**: This guide * **Examples**: Section 9 * **Roadmap**: Section 10 * * * **Document Version**: 1.0 **Last Updated**: 2025-11-19 **EdgarTools Version**: 4.29.0+ **Contributors**: @dgunning, @mpreiss9, EdgarTools community Back to top --- # Multi-Period Analysis - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/latest/xbrl/guides/multi-period-analysis/#multi-period-financial-analysis-with-xbrls) Multi-Period Financial Analysis with XBRLS ========================================== Overview -------- Multi-period financial analysis allows you to compare a company's performance across multiple years or quarters. The `XBRLS` class in edgartools makes this easy by automatically stitching together financial statements from multiple SEC filings. ### Why Use Multi-Period Analysis? Financial analysts need to see trends over time: - **Revenue growth** over 3-5 years - **Margin expansion or compression** - **Balance sheet evolution** - **Cash flow patterns** ### When to Use XBRLS vs Single XBRL | Use Case | Tool | Why | | --- | --- | --- | | Analyze current quarter | `XBRL.from_filing()` | One filing, faster | | Compare 2+ periods | `XBRLS.from_filings()` | Multi-filing stitching | | Historical trends (3-5 years) | `XBRLS.from_filings()` | Handles concept changes | | Quick annual comparison | `Company.income_statement()` | EntityFacts API (simpler) | **Key Difference**: XBRLS works with individual filings and stitches them together, preserving the original XBRL structure. The Company API uses the EntityFacts API, which is pre-aggregated by the SEC but may have different period selections. Quick Example ------------- Here's how to analyze Apple's revenue trend over 3 years: `from edgar import Company from edgar.xbrl import XBRLS # Get the last 3 annual filings company = Company("AAPL") filings = company.get_filings(form="10-K").head(3) # Create XBRLS object (automatically stitches statements) xbrls = XBRLS.from_filings(filings) # Access stitched income statement income = xbrls.statements.income_statement() print(income) # Or convert to DataFrame for analysis df = income.to_dataframe() print(df[['Revenue']])` This automatically: - Parses XBRL from all 3 filings - Aligns periods correctly - Handles concept name changes between years - Creates a unified view Getting Started with XBRLS -------------------------- ### Creating an XBRLS Object There are two ways to create an XBRLS object: **Method 1: From Filings (Recommended)** `from edgar import Company from edgar.xbrl import XBRLS # Get multiple filings company = Company("MSFT") filings = company.get_filings(form="10-K").latest(3) # Create XBRLS xbrls = XBRLS.from_filings(filings)` **Method 2: From XBRL Objects** `from edgar.xbrl import XBRL, XBRLS # If you already have XBRL objects xbrl_list = [XBRL.from_filing(f) for f in filings] xbrls = XBRLS.from_xbrl_objects(xbrl_list)` ### Understanding What XBRLS Does When you create an XBRLS object, it: 1. **Collects all periods** from each filing 2. **Identifies optimal periods** (e.g., fiscal year-ends) 3. **Normalizes concept names** (e.g., "Total Revenue" vs "Net Sales") 4. **Aligns values** across periods 5. **Fills gaps** when a line item appears in some years but not others Accessing Stitched Statements ----------------------------- The `statements` property provides a simple interface to all statement types: `# Get stitched statements balance_sheet = xbrls.statements.balance_sheet() income_statement = xbrls.statements.income_statement() cash_flow = xbrls.statements.cash_flow_statement() # Print statements (uses rich formatting) print(balance_sheet) print(income_statement)` **Available statement methods:** - `balance_sheet()` - Assets, Liabilities, Equity - `income_statement()` - Revenue, Expenses, Net Income - `cash_flow_statement()` - Operating, Investing, Financing Cash Flows - `statement_of_equity()` - Changes in shareholders' equity - `comprehensive_income()` - Other comprehensive income items ### How Stitching Works Conceptually Think of stitching as creating a unified view across filings: `Filing 1 (2024 10-K) Filing 2 (2023 10-K) Filing 3 (2022 10-K) ------------------ ------------------ ------------------ Revenue: $100M (2024) Revenue: $85M (2023) Net Sales: $70M (2022) COGS: $60M COGS: $50M COGS: $42M ------------------ ------------------ ------------------ XBRLS Stitching Process ↓ Unified Statement (Most Recent → Oldest) ---------------------------------------- Revenue: $100M (2024) | $85M (2023) | $70M (2022) COGS: $60M | $50M | $42M` Notice: - "Net Sales" in 2022 was recognized as "Revenue" (concept normalization) - Periods are aligned by fiscal year-end - Labels use the most recent terminology Working with Multi-Period DataFrames ------------------------------------ ### Converting to DataFrame DataFrames are ideal for quantitative analysis: `# Get income statement as DataFrame df = income.to_dataframe() # DataFrame structure: # - Rows: Financial line items (Revenue, Cost of Goods Sold, etc.) # - Columns: Time periods (2024, 2023, 2022) # - Index: Concept names print(df.head())` **Example output:** `2024-09-28 2023-09-30 2022-09-24 Revenue 391035000 383285000 394328000 Cost of Goods Sold 210352000 214137000 223546000 Gross Profit 180683000 169148000 170782000 Operating Expenses 55013000 51345000 51345000 Operating Income 125670000 117803000 119437000` ### Understanding Column Structure Period columns use fiscal year-end dates: `# Examine available periods print(df.columns.tolist()) # ['2024-09-28', '2023-09-30', '2022-09-24'] # Access specific period revenue_2024 = df.loc['Revenue', '2024-09-28'] # Access all periods for a line item revenue_trend = df.loc['Revenue'] print(revenue_trend)` ### Working with Dimensions By default, XBRLS excludes dimensional (segment) data for cleaner consolidated statements. Use the `view` parameter to control this: `# Include dimensional breakdown (e.g., by product line) income = xbrls.statements.income_statement(view="detailed") df = income.to_dataframe() # Now you'll see rows like: # Revenue [Americas] # Revenue [Europe] # Revenue [Asia] # Summary view — non-dimensional totals only income_summary = xbrls.statements.income_statement(view="summary")` The `view` parameter accepts three values: | View | Description | Use When | | --- | --- | --- | | `"standard"` | Face presentation (default) | Consolidated company-level analysis, trend analysis | | `"detailed"` | All dimensional data included | Segment performance, geographic breakdown, product line analysis | | `"summary"` | Non-dimensional totals only | Quick overview of main line items | The legacy `include_dimensions` boolean is still supported (`include_dimensions=True` is equivalent to `view="detailed"`), but `view` is the preferred API. Period Selection ---------------- ### Automatic Optimal Period Selection By default, XBRLS selects the best periods for comparison: `# Automatically selects 3 annual periods xbrls = XBRLS.from_filings(filings) # filings = 3 annual reports income = xbrls.statements.income_statement() # XBRLS picks the fiscal year-end period from each filing # For Apple: Sep 30, Sep 24, Sep 25 (Saturday year-ends)` **How XBRLS Selects Periods:** 1. **Identifies fiscal year-end** from each filing's document period end date 2. **Prefers annual periods** (duration > 300 days) over quarterly 3. **Sorts newest first** for trend analysis 4. **De-duplicates** periods that appear in multiple filings ### Controlling Period Count Use `max_periods` to control how many periods appear: `# Get 5 years instead of 3 filings = company.get_filings(form="10-K").head(5) xbrls = XBRLS.from_filings(filings) # Limit to 5 periods even if more are available income = xbrls.statements.income_statement(max_periods=5) # Or get all available periods income = xbrls.statements.income_statement(max_periods=10)` ### Quarterly Analysis For quarterly trends, use 10-Q filings: `# Get last 8 quarters filings = company.get_filings(form="10-Q").head(8) xbrls = XBRLS.from_filings(filings) # Quarterly income statement income = xbrls.statements.income_statement(max_periods=8) print(income)` ### Manual Period Inspection To see what periods are available: `# Get all available periods periods = xbrls.get_periods() for period in periods: print(f"Type: {period['type']}") print(f"Label: {period['label']}") if period['type'] == 'duration': print(f"Duration: {period['days']} days") print() # Get just the end dates end_dates = xbrls.get_period_end_dates() print(end_dates) # ['2024-09-28', '2023-09-30', '2022-09-24']` Common Use Cases ---------------- ### 1\. Revenue Trend Analysis Track revenue growth over time: `from edgar import Company from edgar.xbrl import XBRLS company = Company("AAPL") filings = company.get_filings(form="10-K").head(5) xbrls = XBRLS.from_filings(filings) # Get income statement income = xbrls.statements.income_statement(max_periods=5) df = income.to_dataframe() # Extract revenue trend revenue = df.loc['Revenue'] print(revenue) # Calculate year-over-year growth yoy_growth = revenue.pct_change() * 100 print("\nYear-over-Year Growth:") print(yoy_growth)` ### 2\. Margin Analysis Over Time Compare profitability trends: `# Get income statement df = income.to_dataframe() # Calculate gross margin for each period revenue = df.loc['Revenue'] gross_profit = df.loc['Gross Profit'] gross_margin = (gross_profit / revenue) * 100 print("Gross Margin Trend:") print(gross_margin) # Operating margin operating_income = df.loc['Operating Income'] operating_margin = (operating_income / revenue) * 100 print("\nOperating Margin Trend:") print(operating_margin)` ### 3\. Balance Sheet Evolution Track how balance sheet composition changes: `# Get balance sheet balance = xbrls.statements.balance_sheet(max_periods=5) df = balance.to_dataframe() # Asset composition total_assets = df.loc['Assets'] cash = df.loc['Cash and Cash Equivalents'] cash_ratio = (cash / total_assets) * 100 print("Cash as % of Total Assets:") print(cash_ratio) # Leverage analysis total_liabilities = df.loc['Liabilities'] equity = df.loc['Stockholders Equity'] debt_to_equity = total_liabilities / equity print("\nDebt-to-Equity Ratio:") print(debt_to_equity)` ### 4\. Cash Flow Pattern Analysis Understand cash generation and usage: `# Get cash flow statement cash_flow = xbrls.statements.cash_flow_statement(max_periods=5) df = cash_flow.to_dataframe() # Operating cash flow trend operating_cf = df.loc['Net Cash Provided by Operating Activities'] print("Operating Cash Flow:") print(operating_cf) # Free cash flow (Operating CF - Capex) capex = df.loc['Capital Expenditures'] free_cash_flow = operating_cf + capex # capex is negative print("\nFree Cash Flow:") print(free_cash_flow) # Cash conversion ratio net_income = income.to_dataframe().loc['Net Income'] cash_conversion = (operating_cf / net_income) * 100 print("\nCash Conversion (Operating CF / Net Income):") print(cash_conversion)` ### 5\. Year-over-Year Comparative Analysis Compare specific line items across years: `import pandas as pd # Get 3 years of data filings = company.get_filings(form="10-K").head(3) xbrls = XBRLS.from_filings(filings) # Create comparison DataFrame income_df = xbrls.statements.income_statement().to_dataframe() # Select key metrics key_metrics = [ 'Revenue', 'Gross Profit', 'Operating Income', 'Net Income' ] comparison = income_df.loc[key_metrics] # Add year-over-year changes for i in range(len(comparison.columns) - 1): current_col = comparison.columns[i] prior_col = comparison.columns[i + 1] change_col = f"{current_col[:4]} vs {prior_col[:4]}" comparison[change_col] = ( (comparison[current_col] - comparison[prior_col]) / comparison[prior_col] * 100 ) print(comparison)` Comparison: XBRLS vs Company API -------------------------------- Both XBRLS and the Company API can provide multi-period statements, but they serve different purposes: ### Feature Comparison | Feature | XBRLS | Company.income\_statement() | | --- | --- | --- | | **Data Source** | Individual XBRL filings | EntityFacts API | | **Setup Complexity** | More code | One-liner | | **Flexibility** | High (custom periods) | Medium (predefined periods) | | **Period Selection** | Filing-based | API-aggregated | | **Concept Stitching** | Automatic | Pre-aggregated by SEC | | **Speed** | Slower (parsing XBRL) | Faster (JSON API) | | **Dimensions** | Full control | Limited access | | **Offline Use** | Possible with caching | Requires API access | | **Best For** | Deep analysis, custom periods | Quick lookups, standard views | ### When to Use XBRLS Use XBRLS when you need: - **Full control over period selection** - **Access to filing-specific details** - **Custom stitching logic** - **Dimensional segment analysis** - **To work with specific filings** (e.g., amended returns) Example: `from edgar.xbrl import XBRLS # Full control over which filings filings = company.get_filings(form="10-K", filing_date="2020-01-01:2024-12-31").head(4) xbrls = XBRLS.from_filings(filings) income = xbrls.statements.income_statement()` ### When to Use Company API Use Company API when you need: - **Quick standard views** - **Simple multi-year comparisons** - **Less code** - **Faster performance** Example: `from edgar import Company # Simple and fast company = Company("AAPL") income = company.income_statement(period='annual', periods=5) print(income)` ### Hybrid Approach You can use both for different purposes: `from edgar import Company company = Company("AAPL") # Quick check with Company API income_quick = company.income_statement(period='annual', periods=3) print("Quick view:", income_quick) # Deep dive with XBRLS filings = company.get_filings(form="10-K").head(5) xbrls = XBRLS.from_filings(filings) income_detailed = xbrls.statements.income_statement(max_periods=5) df = income_detailed.to_dataframe() # Now do custom analysis # ...` Troubleshooting --------------- ### Missing Periods **Problem**: Some periods are missing from stitched statements `# Check available periods periods = xbrls.get_periods() print(f"Found {len(periods)} periods") for p in periods: print(p) # Check if filings have XBRL data for xbrl in xbrls.xbrl_list: print(f"Entity: {xbrl.entity_name}") print(f"Period: {xbrl.period_of_report}") print(f"Statements: {len(xbrl.get_all_statements())}")` **Solutions:** - Ensure filings have XBRL data (pre-2009 filings may not) - Check that filings are the same form type (don't mix 10-K and 10-Q) - Filter amendments: `filings.filter(amendments=False)` ### Stitching Errors **Problem**: Statement fails to stitch or shows unexpected values `# Check individual XBRL objects first for xbrl in xbrls.xbrl_list: print(f"\n{xbrl.entity_name} - {xbrl.period_of_report}") try: stmt = xbrl.statements.income_statement() print(stmt) except Exception as e: print(f"Error: {e}")` **Common causes:** - Company changed fiscal year-end - Different statement structures across years - Missing required concepts in some years **Solution:** Use standardization metadata (enabled by default): `# Standardization adds standard_concept metadata for cross-company analysis income = xbrls.statements.income_statement(standard=True) # Labels always show original company presentation # Use standard_concept column for filtering/aggregation df = income.to_dataframe() print(df[['label', 'standard_concept']].head())` ### Concept Alignment Issues **Problem**: Need to compare similar line items across companies Use the `standard_concept` column to identify equivalent concepts: `# Get DataFrame with standard_concept metadata df = income.to_dataframe() # Filter by standard concept revenue_rows = df[df['standard_concept'] == 'Revenue'] # Aggregate by standard concept for comparison standardized = df.groupby('standard_concept')[['2024-09-30', '2023-09-30']].sum()` > **Note**: Labels preserve the company's original presentation. The `standard_concept` column maps each line item to a standard category for programmatic analysis. ### Performance Tips **Problem**: Stitching is slow for many filings `# 1. Reduce number of periods income = xbrls.statements.income_statement(max_periods=3) # Instead of 10 # 2. Filter amendments before creating XBRLS filings = company.get_filings(form="10-K").filter(amendments=False).head(3) # 3. Use caching for repeated access # Statements are cached automatically within XBRLS object income = xbrls.statements.income_statement() # First call: slow income = xbrls.statements.income_statement() # Second call: fast (cached) # 4. For bulk analysis, create XBRLS once and reuse for statement_type in ['IncomeStatement', 'BalanceSheet', 'CashFlowStatement']: stmt = xbrls.statements[statement_type] # ... analyze ...` Advanced Topics --------------- ### Querying Stitched Facts For advanced analysis, you can query the underlying facts: `# Query across all filings query = xbrls.query(max_periods=5) # Filter to specific concepts revenue_facts = query.by_standardized_concept("Revenue").execute() # Convert to DataFrame for analysis df = query.to_dataframe() # Filter to concepts across all periods consistent_facts = query.across_periods(min_periods=5).execute()` ### Trend Analysis `# Setup trend analysis for specific concept trend_query = xbrls.query().trend_analysis("Revenue") # Get results sorted by period results = trend_query.execute() # Or get as DataFrame with periods as columns trend_df = trend_query.to_trend_dataframe() print(trend_df)` ### Custom Period Selection `# Get statement data with custom period control statement_data = xbrls.get_statement( statement_type='IncomeStatement', max_periods=5, standard=True, use_optimal_periods=True ) # Examine period structure print(statement_data['periods']) # Work with raw data for item in statement_data['statement_data']: print(f"{item['label']}: {item['values']}")` Best Practices -------------- ### 1\. Always Filter Amendments Amendments can cause duplicate periods: `# GOOD filings = company.get_filings(form="10-K").filter(amendments=False).head(5) # AVOID filings = company.get_filings(form="10-K").head(5) # May include amendments` ### 2\. Use Consistent Form Types Don't mix annual and quarterly filings: `# GOOD: All 10-K filings_annual = company.get_filings(form="10-K").head(5) # GOOD: All 10-Q filings_quarterly = company.get_filings(form="10-Q").head(8) # AVOID: Mixed forms filings_mixed = company.get_filings(form=["10-K", "10-Q"]).head(10)` ### 3\. Check Period Alignment Always verify periods align as expected: `xbrls = XBRLS.from_filings(filings) # Check periods before analysis end_dates = xbrls.get_period_end_dates() print("Analyzing periods:", end_dates) # Should be consistent fiscal year-ends # e.g., all December 31 or all September 30` ### 4\. Handle Missing Data Not all line items appear in all periods: `df = income.to_dataframe() # Check for missing values print("\nMissing data by period:") print(df.isnull().sum()) # Fill missing values if appropriate df_filled = df.fillna(0) # Or use forward-fill: df.ffill()` ### 5\. Validate Results Cross-check with SEC filings: `# Print statement to visually verify print(income) # Check against filing filing = filings[0] print(f"\nCompare with: {filing.filing_date}") print(filing.homepage_url) # Verify key metrics df = income.to_dataframe() revenue = df.loc['Revenue'].iloc[0] print(f"Revenue (most recent): ${revenue:,.0f}")` Related Documentation --------------------- * **[Dimension Handling](https://edgartools.readthedocs.io/en/latest/xbrl/guides/dimension-handling.md) ** - Working with segment data * **[Standardization Concepts](https://edgartools.readthedocs.io/en/latest/xbrl/guides/standardization-concepts.md) ** - How concept normalization works * **[XBRL Basics](https://edgartools.readthedocs.io/en/latest/xbrl/guides/xbrl-basics.md) ** - Understanding XBRL structure * **[Company API Reference](https://edgartools.readthedocs.io/en/latest/api/company/) ** - Alternative approach using EntityFacts Summary ------- Multi-period analysis with XBRLS enables powerful trend analysis: **Key Takeaways:** - Use `XBRLS.from_filings()` to create multi-period view - Access statements via `xbrls.statements.income_statement()` - Convert to DataFrame with `.to_dataframe()` for analysis - Control periods with `max_periods` parameter - Always filter amendments for cleaner data - Use standardization (enabled by default) for consistent labels **Quick Reference:** `from edgar import Company from edgar.xbrl import XBRLS # Setup company = Company("AAPL") filings = company.get_filings(form="10-K").filter(amendments=False).head(5) xbrls = XBRLS.from_filings(filings) # Access statements income = xbrls.statements.income_statement(max_periods=5) balance = xbrls.statements.balance_sheet(max_periods=5) cash_flow = xbrls.statements.cash_flow_statement(max_periods=5) # Convert to DataFrame df = income.to_dataframe() # Analyze revenue_trend = df.loc['Revenue'] print(revenue_trend.pct_change() * 100)` For quick lookups, consider the Company API: `# Simpler alternative for standard views income = company.income_statement(period='annual', periods=5)` Choose XBRLS when you need full control and deep analysis. Use Company API for quick standard views. Back to top --- # Multi-Period Analysis - EdgarTools - Python Library for SEC Data Analysis [Skip to content](https://edgartools.readthedocs.io/en/stable/xbrl/guides/multi-period-analysis/#multi-period-financial-analysis-with-xbrls) Multi-Period Financial Analysis with XBRLS ========================================== Overview -------- Multi-period financial analysis allows you to compare a company's performance across multiple years or quarters. The `XBRLS` class in edgartools makes this easy by automatically stitching together financial statements from multiple SEC filings. ### Why Use Multi-Period Analysis? Financial analysts need to see trends over time: - **Revenue growth** over 3-5 years - **Margin expansion or compression** - **Balance sheet evolution** - **Cash flow patterns** ### When to Use XBRLS vs Single XBRL | Use Case | Tool | Why | | --- | --- | --- | | Analyze current quarter | `XBRL.from_filing()` | One filing, faster | | Compare 2+ periods | `XBRLS.from_filings()` | Multi-filing stitching | | Historical trends (3-5 years) | `XBRLS.from_filings()` | Handles concept changes | | Quick annual comparison | `Company.income_statement()` | EntityFacts API (simpler) | **Key Difference**: XBRLS works with individual filings and stitches them together, preserving the original XBRL structure. The Company API uses the EntityFacts API, which is pre-aggregated by the SEC but may have different period selections. Quick Example ------------- Here's how to analyze Apple's revenue trend over 3 years: `from edgar import Company from edgar.xbrl import XBRLS # Get the last 3 annual filings company = Company("AAPL") filings = company.get_filings(form="10-K").head(3) # Create XBRLS object (automatically stitches statements) xbrls = XBRLS.from_filings(filings) # Access stitched income statement income = xbrls.statements.income_statement() print(income) # Or convert to DataFrame for analysis df = income.to_dataframe() print(df[['Revenue']])` This automatically: - Parses XBRL from all 3 filings - Aligns periods correctly - Handles concept name changes between years - Creates a unified view Getting Started with XBRLS -------------------------- ### Creating an XBRLS Object There are two ways to create an XBRLS object: **Method 1: From Filings (Recommended)** `from edgar import Company from edgar.xbrl import XBRLS # Get multiple filings company = Company("MSFT") filings = company.get_filings(form="10-K").latest(3) # Create XBRLS xbrls = XBRLS.from_filings(filings)` **Method 2: From XBRL Objects** `from edgar.xbrl import XBRL, XBRLS # If you already have XBRL objects xbrl_list = [XBRL.from_filing(f) for f in filings] xbrls = XBRLS.from_xbrl_objects(xbrl_list)` ### Understanding What XBRLS Does When you create an XBRLS object, it: 1. **Collects all periods** from each filing 2. **Identifies optimal periods** (e.g., fiscal year-ends) 3. **Normalizes concept names** (e.g., "Total Revenue" vs "Net Sales") 4. **Aligns values** across periods 5. **Fills gaps** when a line item appears in some years but not others Accessing Stitched Statements ----------------------------- The `statements` property provides a simple interface to all statement types: `# Get stitched statements balance_sheet = xbrls.statements.balance_sheet() income_statement = xbrls.statements.income_statement() cash_flow = xbrls.statements.cash_flow_statement() # Print statements (uses rich formatting) print(balance_sheet) print(income_statement)` **Available statement methods:** - `balance_sheet()` - Assets, Liabilities, Equity - `income_statement()` - Revenue, Expenses, Net Income - `cash_flow_statement()` - Operating, Investing, Financing Cash Flows - `statement_of_equity()` - Changes in shareholders' equity - `comprehensive_income()` - Other comprehensive income items ### How Stitching Works Conceptually Think of stitching as creating a unified view across filings: `Filing 1 (2024 10-K) Filing 2 (2023 10-K) Filing 3 (2022 10-K) ------------------ ------------------ ------------------ Revenue: $100M (2024) Revenue: $85M (2023) Net Sales: $70M (2022) COGS: $60M COGS: $50M COGS: $42M ------------------ ------------------ ------------------ XBRLS Stitching Process ↓ Unified Statement (Most Recent → Oldest) ---------------------------------------- Revenue: $100M (2024) | $85M (2023) | $70M (2022) COGS: $60M | $50M | $42M` Notice: - "Net Sales" in 2022 was recognized as "Revenue" (concept normalization) - Periods are aligned by fiscal year-end - Labels use the most recent terminology Working with Multi-Period DataFrames ------------------------------------ ### Converting to DataFrame DataFrames are ideal for quantitative analysis: `# Get income statement as DataFrame df = income.to_dataframe() # DataFrame structure: # - Rows: Financial line items (Revenue, Cost of Goods Sold, etc.) # - Columns: Time periods (2024, 2023, 2022) # - Index: Concept names print(df.head())` **Example output:** `2024-09-28 2023-09-30 2022-09-24 Revenue 391035000 383285000 394328000 Cost of Goods Sold 210352000 214137000 223546000 Gross Profit 180683000 169148000 170782000 Operating Expenses 55013000 51345000 51345000 Operating Income 125670000 117803000 119437000` ### Understanding Column Structure Period columns use fiscal year-end dates: `# Examine available periods print(df.columns.tolist()) # ['2024-09-28', '2023-09-30', '2022-09-24'] # Access specific period revenue_2024 = df.loc['Revenue', '2024-09-28'] # Access all periods for a line item revenue_trend = df.loc['Revenue'] print(revenue_trend)` ### Working with Dimensions By default, XBRLS excludes dimensional (segment) data for cleaner consolidated statements. Use the `view` parameter to control this: `# Include dimensional breakdown (e.g., by product line) income = xbrls.statements.income_statement(view="detailed") df = income.to_dataframe() # Now you'll see rows like: # Revenue [Americas] # Revenue [Europe] # Revenue [Asia] # Summary view — non-dimensional totals only income_summary = xbrls.statements.income_statement(view="summary")` The `view` parameter accepts three values: | View | Description | Use When | | --- | --- | --- | | `"standard"` | Face presentation (default) | Consolidated company-level analysis, trend analysis | | `"detailed"` | All dimensional data included | Segment performance, geographic breakdown, product line analysis | | `"summary"` | Non-dimensional totals only | Quick overview of main line items | The legacy `include_dimensions` boolean is still supported (`include_dimensions=True` is equivalent to `view="detailed"`), but `view` is the preferred API. Period Selection ---------------- ### Automatic Optimal Period Selection By default, XBRLS selects the best periods for comparison: `# Automatically selects 3 annual periods xbrls = XBRLS.from_filings(filings) # filings = 3 annual reports income = xbrls.statements.income_statement() # XBRLS picks the fiscal year-end period from each filing # For Apple: Sep 30, Sep 24, Sep 25 (Saturday year-ends)` **How XBRLS Selects Periods:** 1. **Identifies fiscal year-end** from each filing's document period end date 2. **Prefers annual periods** (duration > 300 days) over quarterly 3. **Sorts newest first** for trend analysis 4. **De-duplicates** periods that appear in multiple filings ### Controlling Period Count Use `max_periods` to control how many periods appear: `# Get 5 years instead of 3 filings = company.get_filings(form="10-K").head(5) xbrls = XBRLS.from_filings(filings) # Limit to 5 periods even if more are available income = xbrls.statements.income_statement(max_periods=5) # Or get all available periods income = xbrls.statements.income_statement(max_periods=10)` ### Quarterly Analysis For quarterly trends, use 10-Q filings: `# Get last 8 quarters filings = company.get_filings(form="10-Q").head(8) xbrls = XBRLS.from_filings(filings) # Quarterly income statement income = xbrls.statements.income_statement(max_periods=8) print(income)` ### Manual Period Inspection To see what periods are available: `# Get all available periods periods = xbrls.get_periods() for period in periods: print(f"Type: {period['type']}") print(f"Label: {period['label']}") if period['type'] == 'duration': print(f"Duration: {period['days']} days") print() # Get just the end dates end_dates = xbrls.get_period_end_dates() print(end_dates) # ['2024-09-28', '2023-09-30', '2022-09-24']` Common Use Cases ---------------- ### 1\. Revenue Trend Analysis Track revenue growth over time: `from edgar import Company from edgar.xbrl import XBRLS company = Company("AAPL") filings = company.get_filings(form="10-K").head(5) xbrls = XBRLS.from_filings(filings) # Get income statement income = xbrls.statements.income_statement(max_periods=5) df = income.to_dataframe() # Extract revenue trend revenue = df.loc['Revenue'] print(revenue) # Calculate year-over-year growth yoy_growth = revenue.pct_change() * 100 print("\nYear-over-Year Growth:") print(yoy_growth)` ### 2\. Margin Analysis Over Time Compare profitability trends: `# Get income statement df = income.to_dataframe() # Calculate gross margin for each period revenue = df.loc['Revenue'] gross_profit = df.loc['Gross Profit'] gross_margin = (gross_profit / revenue) * 100 print("Gross Margin Trend:") print(gross_margin) # Operating margin operating_income = df.loc['Operating Income'] operating_margin = (operating_income / revenue) * 100 print("\nOperating Margin Trend:") print(operating_margin)` ### 3\. Balance Sheet Evolution Track how balance sheet composition changes: `# Get balance sheet balance = xbrls.statements.balance_sheet(max_periods=5) df = balance.to_dataframe() # Asset composition total_assets = df.loc['Assets'] cash = df.loc['Cash and Cash Equivalents'] cash_ratio = (cash / total_assets) * 100 print("Cash as % of Total Assets:") print(cash_ratio) # Leverage analysis total_liabilities = df.loc['Liabilities'] equity = df.loc['Stockholders Equity'] debt_to_equity = total_liabilities / equity print("\nDebt-to-Equity Ratio:") print(debt_to_equity)` ### 4\. Cash Flow Pattern Analysis Understand cash generation and usage: `# Get cash flow statement cash_flow = xbrls.statements.cash_flow_statement(max_periods=5) df = cash_flow.to_dataframe() # Operating cash flow trend operating_cf = df.loc['Net Cash Provided by Operating Activities'] print("Operating Cash Flow:") print(operating_cf) # Free cash flow (Operating CF - Capex) capex = df.loc['Capital Expenditures'] free_cash_flow = operating_cf + capex # capex is negative print("\nFree Cash Flow:") print(free_cash_flow) # Cash conversion ratio net_income = income.to_dataframe().loc['Net Income'] cash_conversion = (operating_cf / net_income) * 100 print("\nCash Conversion (Operating CF / Net Income):") print(cash_conversion)` ### 5\. Year-over-Year Comparative Analysis Compare specific line items across years: `import pandas as pd # Get 3 years of data filings = company.get_filings(form="10-K").head(3) xbrls = XBRLS.from_filings(filings) # Create comparison DataFrame income_df = xbrls.statements.income_statement().to_dataframe() # Select key metrics key_metrics = [ 'Revenue', 'Gross Profit', 'Operating Income', 'Net Income' ] comparison = income_df.loc[key_metrics] # Add year-over-year changes for i in range(len(comparison.columns) - 1): current_col = comparison.columns[i] prior_col = comparison.columns[i + 1] change_col = f"{current_col[:4]} vs {prior_col[:4]}" comparison[change_col] = ( (comparison[current_col] - comparison[prior_col]) / comparison[prior_col] * 100 ) print(comparison)` Comparison: XBRLS vs Company API -------------------------------- Both XBRLS and the Company API can provide multi-period statements, but they serve different purposes: ### Feature Comparison | Feature | XBRLS | Company.income\_statement() | | --- | --- | --- | | **Data Source** | Individual XBRL filings | EntityFacts API | | **Setup Complexity** | More code | One-liner | | **Flexibility** | High (custom periods) | Medium (predefined periods) | | **Period Selection** | Filing-based | API-aggregated | | **Concept Stitching** | Automatic | Pre-aggregated by SEC | | **Speed** | Slower (parsing XBRL) | Faster (JSON API) | | **Dimensions** | Full control | Limited access | | **Offline Use** | Possible with caching | Requires API access | | **Best For** | Deep analysis, custom periods | Quick lookups, standard views | ### When to Use XBRLS Use XBRLS when you need: - **Full control over period selection** - **Access to filing-specific details** - **Custom stitching logic** - **Dimensional segment analysis** - **To work with specific filings** (e.g., amended returns) Example: `from edgar.xbrl import XBRLS # Full control over which filings filings = company.get_filings(form="10-K", filing_date="2020-01-01:2024-12-31").head(4) xbrls = XBRLS.from_filings(filings) income = xbrls.statements.income_statement()` ### When to Use Company API Use Company API when you need: - **Quick standard views** - **Simple multi-year comparisons** - **Less code** - **Faster performance** Example: `from edgar import Company # Simple and fast company = Company("AAPL") income = company.income_statement(period='annual', periods=5) print(income)` ### Hybrid Approach You can use both for different purposes: `from edgar import Company company = Company("AAPL") # Quick check with Company API income_quick = company.income_statement(period='annual', periods=3) print("Quick view:", income_quick) # Deep dive with XBRLS filings = company.get_filings(form="10-K").head(5) xbrls = XBRLS.from_filings(filings) income_detailed = xbrls.statements.income_statement(max_periods=5) df = income_detailed.to_dataframe() # Now do custom analysis # ...` Troubleshooting --------------- ### Missing Periods **Problem**: Some periods are missing from stitched statements `# Check available periods periods = xbrls.get_periods() print(f"Found {len(periods)} periods") for p in periods: print(p) # Check if filings have XBRL data for xbrl in xbrls.xbrl_list: print(f"Entity: {xbrl.entity_name}") print(f"Period: {xbrl.period_of_report}") print(f"Statements: {len(xbrl.get_all_statements())}")` **Solutions:** - Ensure filings have XBRL data (pre-2009 filings may not) - Check that filings are the same form type (don't mix 10-K and 10-Q) - Filter amendments: `filings.filter(amendments=False)` ### Stitching Errors **Problem**: Statement fails to stitch or shows unexpected values `# Check individual XBRL objects first for xbrl in xbrls.xbrl_list: print(f"\n{xbrl.entity_name} - {xbrl.period_of_report}") try: stmt = xbrl.statements.income_statement() print(stmt) except Exception as e: print(f"Error: {e}")` **Common causes:** - Company changed fiscal year-end - Different statement structures across years - Missing required concepts in some years **Solution:** Use standardization metadata (enabled by default): `# Standardization adds standard_concept metadata for cross-company analysis income = xbrls.statements.income_statement(standard=True) # Labels always show original company presentation # Use standard_concept column for filtering/aggregation df = income.to_dataframe() print(df[['label', 'standard_concept']].head())` ### Concept Alignment Issues **Problem**: Need to compare similar line items across companies Use the `standard_concept` column to identify equivalent concepts: `# Get DataFrame with standard_concept metadata df = income.to_dataframe() # Filter by standard concept revenue_rows = df[df['standard_concept'] == 'Revenue'] # Aggregate by standard concept for comparison standardized = df.groupby('standard_concept')[['2024-09-30', '2023-09-30']].sum()` > **Note**: Labels preserve the company's original presentation. The `standard_concept` column maps each line item to a standard category for programmatic analysis. ### Performance Tips **Problem**: Stitching is slow for many filings `# 1. Reduce number of periods income = xbrls.statements.income_statement(max_periods=3) # Instead of 10 # 2. Filter amendments before creating XBRLS filings = company.get_filings(form="10-K").filter(amendments=False).head(3) # 3. Use caching for repeated access # Statements are cached automatically within XBRLS object income = xbrls.statements.income_statement() # First call: slow income = xbrls.statements.income_statement() # Second call: fast (cached) # 4. For bulk analysis, create XBRLS once and reuse for statement_type in ['IncomeStatement', 'BalanceSheet', 'CashFlowStatement']: stmt = xbrls.statements[statement_type] # ... analyze ...` Advanced Topics --------------- ### Querying Stitched Facts For advanced analysis, you can query the underlying facts: `# Query across all filings query = xbrls.query(max_periods=5) # Filter to specific concepts revenue_facts = query.by_standardized_concept("Revenue").execute() # Convert to DataFrame for analysis df = query.to_dataframe() # Filter to concepts across all periods consistent_facts = query.across_periods(min_periods=5).execute()` ### Trend Analysis `# Setup trend analysis for specific concept trend_query = xbrls.query().trend_analysis("Revenue") # Get results sorted by period results = trend_query.execute() # Or get as DataFrame with periods as columns trend_df = trend_query.to_trend_dataframe() print(trend_df)` ### Custom Period Selection `# Get statement data with custom period control statement_data = xbrls.get_statement( statement_type='IncomeStatement', max_periods=5, standard=True, use_optimal_periods=True ) # Examine period structure print(statement_data['periods']) # Work with raw data for item in statement_data['statement_data']: print(f"{item['label']}: {item['values']}")` Best Practices -------------- ### 1\. Always Filter Amendments Amendments can cause duplicate periods: `# GOOD filings = company.get_filings(form="10-K").filter(amendments=False).head(5) # AVOID filings = company.get_filings(form="10-K").head(5) # May include amendments` ### 2\. Use Consistent Form Types Don't mix annual and quarterly filings: `# GOOD: All 10-K filings_annual = company.get_filings(form="10-K").head(5) # GOOD: All 10-Q filings_quarterly = company.get_filings(form="10-Q").head(8) # AVOID: Mixed forms filings_mixed = company.get_filings(form=["10-K", "10-Q"]).head(10)` ### 3\. Check Period Alignment Always verify periods align as expected: `xbrls = XBRLS.from_filings(filings) # Check periods before analysis end_dates = xbrls.get_period_end_dates() print("Analyzing periods:", end_dates) # Should be consistent fiscal year-ends # e.g., all December 31 or all September 30` ### 4\. Handle Missing Data Not all line items appear in all periods: `df = income.to_dataframe() # Check for missing values print("\nMissing data by period:") print(df.isnull().sum()) # Fill missing values if appropriate df_filled = df.fillna(0) # Or use forward-fill: df.ffill()` ### 5\. Validate Results Cross-check with SEC filings: `# Print statement to visually verify print(income) # Check against filing filing = filings[0] print(f"\nCompare with: {filing.filing_date}") print(filing.homepage_url) # Verify key metrics df = income.to_dataframe() revenue = df.loc['Revenue'].iloc[0] print(f"Revenue (most recent): ${revenue:,.0f}")` Related Documentation --------------------- * **[Dimension Handling](https://edgartools.readthedocs.io/en/stable/xbrl/guides/dimension-handling.md) ** - Working with segment data * **[Standardization Concepts](https://edgartools.readthedocs.io/en/stable/xbrl/guides/standardization-concepts.md) ** - How concept normalization works * **[XBRL Basics](https://edgartools.readthedocs.io/en/stable/xbrl/guides/xbrl-basics.md) ** - Understanding XBRL structure * **[Company API Reference](https://edgartools.readthedocs.io/en/stable/api/company/) ** - Alternative approach using EntityFacts Summary ------- Multi-period analysis with XBRLS enables powerful trend analysis: **Key Takeaways:** - Use `XBRLS.from_filings()` to create multi-period view - Access statements via `xbrls.statements.income_statement()` - Convert to DataFrame with `.to_dataframe()` for analysis - Control periods with `max_periods` parameter - Always filter amendments for cleaner data - Use standardization (enabled by default) for consistent labels **Quick Reference:** `from edgar import Company from edgar.xbrl import XBRLS # Setup company = Company("AAPL") filings = company.get_filings(form="10-K").filter(amendments=False).head(5) xbrls = XBRLS.from_filings(filings) # Access statements income = xbrls.statements.income_statement(max_periods=5) balance = xbrls.statements.balance_sheet(max_periods=5) cash_flow = xbrls.statements.cash_flow_statement(max_periods=5) # Convert to DataFrame df = income.to_dataframe() # Analyze revenue_trend = df.loc['Revenue'] print(revenue_trend.pct_change() * 100)` For quick lookups, consider the Company API: `# Simpler alternative for standard views income = company.income_statement(period='annual', periods=5)` Choose XBRLS when you need full control and deep analysis. Use Company API for quick standard views. Back to top ---