r/scrapy
Scrapy: An open source web scraping framework for Python
en
Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.
Scrapy is a powerful open source web scraping & crawling framework for Python.
6,082
March 11, 2014
public
all_ads
Total Submissions
2,320
Total Comments
8,825
Earliest Submission
March 28, 2015
Earliest Comment
July 15, 2015
Rank↕ | Word↕ | Occurrences in Subreddit | Total Occurrences | Rate in Subreddit (per 1M words) | Rate in Reddit (per 1M words) | Ratio (Sub Rate / Reddit Rate) |
|---|---|---|---|---|---|---|
| #1 | 84 | 26,923 | 1688.1 | 0.2 | 7519.27 | |
| #2 | 54 | 25,887 | 1085.2 | 0.2 | 5027.26 | |
| #3 | 46 | 25,414 | 924.4 | 0.2 | 4362.19 | |
| #4 | 35 | 23,551 | 703.4 | 0.2 | 3581.61 | |
| #5 | 49 | 36,252 | 984.7 | 0.3 | 3257.49 | |
| #6 | 35 | 50,608 | 703.4 | 0.4 | 1666.74 | |
| #7 | 86 | 146,708 | 1728.3 | 1.2 | 1412.75 | |
| #8 | 35 | 60,502 | 703.4 | 0.5 | 1394.18 | |
| #9 | 97 | 182,431 | 1949.3 | 1.5 | 1281.42 | |
| #10 | 116 | 237,439 | 2331.1 | 2.0 | 1177.40 | |
| #11 | 44 | 116,762 | 884.2 | 1.0 | 908.18 | |
| #12 | 118 | 318,077 | 2371.3 | 2.7 | 894.07 | |
| #13 | 71 | 200,002 | 1426.8 | 1.7 | 855.55 | |
| #14 | 78 | 239,963 | 1567.5 | 2.0 | 783.38 | |
| #15 | 42 | 165,032 | 844.0 | 1.4 | 613.34 | |
| #16 | 86 | 369,424 | 1728.3 | 3.1 | 561.04 | |
| #17 | 55 | 255,422 | 1105.3 | 2.1 | 518.95 | |
| #18 | 93 | 487,454 | 1868.9 | 4.1 | 459.80 | |
| #19 | 31 | 184,241 | 623.0 | 1.5 | 405.50 | |
| #20 | 45 | 291,964 | 904.3 | 2.4 | 371.45 | |
| #21 | 49 | 329,991 | 984.7 | 2.8 | 357.86 | |
| #22 | 75 | 556,421 | 1507.2 | 4.6 | 324.85 | |
| #23 | 61 | 550,511 | 1225.9 | 4.6 | 267.04 | |
| #24 | 281 | 2,723,409 | 5647.0 | 22.7 | 248.66 | |
| #25 | 74 | 727,211 | 1487.1 | 6.1 | 245.24 | |
| #26 | 57 | 573,569 | 1145.5 | 4.8 | 239.50 | |
| #27 | 77 | 828,260 | 1547.4 | 6.9 | 224.05 | |
| #28 | 102 | 1,193,988 | 2049.8 | 10.0 | 205.88 | |
| #29 | 51 | 654,988 | 1024.9 | 5.5 | 187.65 | |
| #30 | 77 | 1,040,010 | 1547.4 | 8.7 | 178.43 | |
| #31 | 52 | 717,093 | 1045.0 | 6.0 | 174.76 | |
| #32 | 69 | 1,058,154 | 1386.6 | 8.8 | 157.15 | |
| #33 | 51 | 860,058 | 1024.9 | 7.2 | 142.91 | |
| #34 | 51 | 913,891 | 1024.9 | 7.6 | 134.49 | |
| #35 | 39 | 711,104 | 783.7 | 5.9 | 132.18 | |
| #36 | 68 | 1,284,673 | 1366.5 | 10.7 | 127.57 | |
| #37 | 43 | 823,195 | 864.1 | 6.9 | 125.89 | |
| #38 | 36 | 904,705 | 723.5 | 7.5 | 95.90 | |
| #39 | 70 | 1,933,359 | 1406.7 | 16.1 | 87.26 | |
| #40 | 51 | 1,616,295 | 1024.9 | 13.5 | 76.04 | |
| #41 | 287 | 9,111,836 | 5767.6 | 76.0 | 75.91 | |
| #42 | 36 | 1,149,679 | 723.5 | 9.6 | 75.46 | |
| #43 | 30 | 1,042,247 | 602.9 | 8.7 | 69.37 | |
| #44 | 103 | 3,789,727 | 2069.9 | 31.6 | 65.50 | |
| #45 | 281 | 11,763,192 | 5647.0 | 98.1 | 57.57 | |
| #46 | 184 | 7,750,568 | 3697.7 | 64.6 | 57.21 | |
| #47 | 52 | 2,258,087 | 1045.0 | 18.8 | 55.50 | |
| #48 | 52 | 2,426,052 | 1045.0 | 20.2 | 51.66 | |
| #49 | 43 | 2,079,095 | 864.1 | 17.3 | 49.84 | |
| #50 | 38 | 1,878,037 | 763.7 | 15.7 | 48.76 |