thibaultmol/smallweb
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
Collection of small websites made by actual people
2,805 commits 1 branch 0 tags 16 MiB
  • Shell 100%
Find a file
Exact
Thibault Molleman 017d464711 init commit
2026-06-17 10:39:36 +02:00
.github Update link to guidelines in pull request template 2026-04-10 02:07:44 -04:00
maintenance fix: Fix sed -i for BSD/macOS compatibility 2025-07-01 18:36:48 +02:00
LICENSE init 2023-09-07 07:38:18 -07:00
README.md init commit 2026-06-17 10:39:36 +02:00
smallcomic.original.txt init commit 2026-06-17 10:39:36 +02:00
smallcomic.txt init commit 2026-06-17 10:39:36 +02:00
smallcomic.verified.txt init commit 2026-06-17 10:39:36 +02:00
smallweb.original.txt init commit 2026-06-17 10:39:36 +02:00
smallweb.txt init commit 2026-06-17 10:39:36 +02:00
smallweb.verified.txt init commit 2026-06-17 10:39:36 +02:00
smallyt.original.txt init commit 2026-06-17 10:39:36 +02:00
smallyt.txt init commit 2026-06-17 10:39:36 +02:00
smallyt.verified.txt init commit 2026-06-17 10:39:36 +02:00

README.md

Small Web list

This is a fork of kagisearch/smallweb.

The purpose of this Small Web repo is to collect the RSS/Atom feeds of small, individual websites — personal blogs, independent creators — rather than big commercial output. And making sure that all urls are human-verified.

The repo over at Kagi was a great start but eventually the ceo started vibe-adding domains that absolutely don't belong in such a list. So I've decided to fork it and remove any lines he's added since 2025-11-01.

The plan for this fork

So what I've done is kept the current 'upstream' files in the .original.txt files for now. The current .txt files are the original ones but those lines from Vlad stripped out since that date. So that file SHOULD MOSTLY BE FINE. BUT because I wouldn't be surprised if accepting PR's also was pretty loose at some point, I figured it wouldn't be a bad idea to just start a manual review process. So the .verified.txt files are those. They're empty at the start of making this fork but plan is to fill them overtime as I go over the urls in the .txt file. Eventually once everything is verified, then the .verified.txt files will go and we'll stay with the normal .txt files. So if you have an application that uses the Kagi smallweb .txt files, you can use this repo instead. (I wouldn't use the .verified.txt files cause those will be much smaller and eventually get deleted anyway once everything has been verified)

What I've also done is removed any of the app/docker stuff from the original repo. I myself do not plan to maintain a viewing app or combined rss feed. I'm not against someone contributing that BUT I'm more in favor of you maintaining your own app and this repo sticks to maintaining the list.

What has yet to be done

  • Maybe reconsider some of the guidelines for inclusion? (already made some changes like removing the recency criteria)
  • Consider removing the non-english requirement, this seems like a very American thing to have been decided the original repo. BUT it probably wouldn't be a bad idea to have languages listed or something although that would then result in a csv instead of a txt.... 🤷 tbd (writing this now: maybe we could have a csv and then generate various .txt's based on certain criteria?)
  • Remove yt channels that have now gone above the 100k sub requirement.

⚠️ Guidelines for site inclusion to the list ⚠️

Add a new personal blog RSS/Atom feed to the list. Rules:

  • If submitting your own website, you must add at least 2 other sites that are not yours (and are not in list yet) in the same commit.
  • Locate and submit the RSS/Atom feed of the website. Place in the file so that it remains sorted.
  • Content must be in English (currently, other languages are not accepted).
  • No illegal or NSFW content.
  • No auto generated, LLM generated or spam content. (see below for the nuance)
  • Only personal blogs may be submitted (no multi-author blogs).
  • The site must not contain any forms of advertisements or undisclosed affiliate links
  • Site should not have popups (newsletter signup, cookie selection, etc)
  • A Youtube channel must not post more than twice a week.
  • A Youtube channel must have fewer than 100,000 subscribers.

For comics:

  • Must be independently created art (no AI generated content)
  • RSS feed must show the full comic in the feed
  • No commercial syndicated comics

Add website RSS feed

Hint: To extract the RSS link from a YouTube channel, you can use this tool.

Add YouTube channel RSS feed

Add Comic RSS feed

Remove a site or a channel

Remove a website if :

  • It does not adhere to the above guidelines
  • In the removal request, state which guideline does it break

Clicking "Remove website" will edit small web list in new tab, where you can locate and remove the website feed in question. Make sure to add in comments the reason for removal.

Remove website

Remove channel

Remove comic

Small web is beautiful

What is Small Web exactly? Recommend reading:

Info

smallweb.txt - Contains the feeds of indexed blogs

smallyt.txt - Contains the feeds of indexed YouTube channels

smallcomic.txt - Contains the feeds of indexed independent comics

yt_rejected.txt - Contains the list of YouTube channels that were reviewed (in an automated way) and rejected

Sources

Small web

The original list of small web blogs has been assembled from various sources including:

YouTube channels

The seed list for YouTube channels has been assembled from these HN discussions.

Useful commands

Show duplicate domains:

awk -F/ '{print $3}' smallweb.txt | sort | uniq -d | while read domain; do echo "$domain"; grep "$domain" smallweb.txt; echo ""; done