Creating a repository indexer
Currently, to utilize GitHub Pages on our current free plan, a repository must remain publicly visible unless you have an enterprise account, which we don’t. However, I’d advise against making it public for your specific needs unless you also intend to use Pages. Currently, GitHub doesn’t offer a feature on the free plan for private pages that’s easily accessible and simple to maintain compared to conventional web hosting. Yet, the solution to achieve our objective isn’t overly complicated. Let’s explore how to create a crawler that indexes repositories and evaluates their content:
- Our first step is to design a workflow for the indexer to do this. Design a workflow that does the following:
- Runs on a set schedule
- Can be manually triggered
- The workflow will collect data from all other repositories using the crawler pattern.
- Utilize the GitHub API to fetch repository information.
- Checkout the target...