General Information
As of February 8, 2024
r/scrapy
Scrapy: An open source web scraping framework for Python
en
Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.
Scrapy
Scrapy is a powerful open source web scraping & crawling framework for Python.
Community
- Scrapy on GitHub
- Scrapy on StackOverflow
- [@scrapyproject on Twitter](http://twitter...
6,082
March 11, 2014
public
all_ads
Restricted Submissions
Community Reviewed
Total Submissions
2,320
Total Comments
8,825
Earliest Submission
March 28, 2015
Earliest Comment
July 15, 2015
Metrics
Activity Over Time
totalSubmissions
totalComments
Submission Flags
over 18
locked
score hidden
text submission
stickied
distinguished user
gilded
saved
collapsed
quarantine
Comment Flags
gilded
distinguished user
collapsed
locked
score hidden
Submission Score Distribution
< 0
0
1-10
11-50
51-100
> 100
Comment Score Distribution
< 0
0
1-10
11-50
51-100
> 100
Title Length
0
1-100
101-500
501-1000
>1000
Submission Text Length
0
1-100
101-500
501-1000
>1000
Comments per Submission
< 0
0
1-10
11-50
51-100
> 100
Most Distinctive Words
Year
2024
Total Words in Subreddit
49,761
Total Words in Reddit
119,924,722,855
Rank↕ | Word↕ | Occurrences in Subreddit | Total Occurrences | Rate in Subreddit (per 1M words) | Rate in Reddit (per 1M words) | Ratio (Sub Rate / Reddit Rate) |
|---|---|---|---|---|---|---|
| #1 | 84 | 26,923 | 1688.1 | 0.2 | 7519.27 | |
| #2 | 54 | 25,887 | 1085.2 | 0.2 | 5027.26 | |
| #3 | 46 | 25,414 | 924.4 | 0.2 | 4362.19 | |
| #4 | 35 | 23,551 | 703.4 | 0.2 | 3581.61 | |
| #5 | 49 | 36,252 | 984.7 | 0.3 | 3257.49 | |
| #6 | 35 | 50,608 | 703.4 | 0.4 | 1666.74 | |
| #7 | 86 | 146,708 | 1728.3 | 1.2 | 1412.75 | |
| #8 | 35 | 60,502 | 703.4 | 0.5 | 1394.18 | |
| #9 | 97 | 182,431 | 1949.3 | 1.5 | 1281.42 | |
| #10 | 116 | 237,439 | 2331.1 | 2.0 | 1177.40 | |
| #11 | 44 | 116,762 | 884.2 | 1.0 | 908.18 | |
| #12 | 118 | 318,077 | 2371.3 | 2.7 | 894.07 | |
| #13 | 71 | 200,002 | 1426.8 | 1.7 | 855.55 | |
| #14 | 78 | 239,963 | 1567.5 | 2.0 | 783.38 | |
| #15 | 42 | 165,032 | 844.0 | 1.4 | 613.34 | |
| #16 | 86 | 369,424 | 1728.3 | 3.1 | 561.04 | |
| #17 | 55 | 255,422 | 1105.3 | 2.1 | 518.95 | |
| #18 | 93 | 487,454 | 1868.9 | 4.1 | 459.80 | |
| #19 | 31 | 184,241 | 623.0 | 1.5 | 405.50 | |
| #20 | 45 | 291,964 | 904.3 | 2.4 | 371.45 | |
| #21 | 49 | 329,991 | 984.7 | 2.8 | 357.86 | |
| #22 | 75 | 556,421 | 1507.2 | 4.6 | 324.85 | |
| #23 | 61 | 550,511 | 1225.9 | 4.6 | 267.04 | |
| #24 | 281 | 2,723,409 | 5647.0 | 22.7 | 248.66 | |
| #25 | 74 | 727,211 | 1487.1 | 6.1 | 245.24 | |
| #26 | 57 | 573,569 | 1145.5 | 4.8 | 239.50 | |
| #27 | 77 | 828,260 | 1547.4 | 6.9 | 224.05 | |
| #28 | 102 | 1,193,988 | 2049.8 | 10.0 | 205.88 | |
| #29 | 51 | 654,988 | 1024.9 | 5.5 | 187.65 | |
| #30 | 77 | 1,040,010 | 1547.4 | 8.7 | 178.43 | |
| #31 | 52 | 717,093 | 1045.0 | 6.0 | 174.76 | |
| #32 | 69 | 1,058,154 | 1386.6 | 8.8 | 157.15 | |
| #33 | 51 | 860,058 | 1024.9 | 7.2 | 142.91 | |
| #34 | 51 | 913,891 | 1024.9 | 7.6 | 134.49 | |
| #35 | 39 | 711,104 | 783.7 | 5.9 | 132.18 | |
| #36 | 68 | 1,284,673 | 1366.5 | 10.7 | 127.57 | |
| #37 | 43 | 823,195 | 864.1 | 6.9 | 125.89 | |
| #38 | 36 | 904,705 | 723.5 | 7.5 | 95.90 | |
| #39 | 70 | 1,933,359 | 1406.7 | 16.1 | 87.26 | |
| #40 | 51 | 1,616,295 | 1024.9 | 13.5 | 76.04 | |
| #41 | 287 | 9,111,836 | 5767.6 | 76.0 | 75.91 | |
| #42 | 36 | 1,149,679 | 723.5 | 9.6 | 75.46 | |
| #43 | 30 | 1,042,247 | 602.9 | 8.7 | 69.37 | |
| #44 | 103 | 3,789,727 | 2069.9 | 31.6 | 65.50 | |
| #45 | 281 | 11,763,192 | 5647.0 | 98.1 | 57.57 | |
| #46 | 184 | 7,750,568 | 3697.7 | 64.6 | 57.21 | |
| #47 | 52 | 2,258,087 | 1045.0 | 18.8 | 55.50 | |
| #48 | 52 | 2,426,052 | 1045.0 | 20.2 | 51.66 | |
| #49 | 43 | 2,079,095 | 864.1 | 17.3 | 49.84 | |
| #50 | 38 | 1,878,037 | 763.7 | 15.7 | 48.76 |
Showing 50 of 50 words