Skip to content

Commit 7a17256

Browse files
committed
update readme
1 parent 80b40cb commit 7a17256

File tree

2 files changed

+17
-9
lines changed

2 files changed

+17
-9
lines changed

README.md

+17-9
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,25 @@
11
# Go Webcrawler
22

33
[![Go](https://github.com/Acollie/Go-Webcrawler/actions/workflows/go.yml/badge.svg)](https://github.com/Acollie/Go-Webcrawler/actions/workflows/go.yml)
4-
## Description
54

6-
## Testing
5+
## Overview
76

8-
### Unit Tests
7+
![Overview of architecture](/assets/overview.png "Overview")
98

10-
```bash go test ./...```
9+
### SQS Queue
10+
11+
The SQS queue is used to store the URLs that need to be crawled.
12+
13+
### DynamoDB
14+
15+
DynamoDB is used to store the URLs that have been crawled and the URLs that have been found on the page.
1116

12-
### Integration Tests
17+
### Neptune / Neo4k
18+
19+
Neptune is used to store the relationships between the URLs that have been found on the page.
20+
21+
### Tests
22+
23+
The tests are written in Go and can be run using the following command:
24+
```bash go test ./...```
1325

14-
1. ```docker-compose up -d```
15-
2. ``` aws --endpoint-url=http://localhost:4570 sqs create-queue --queue-name test-queue``` use the results from this to
16-
add to the `.env`
17-
3. ```bash go test ./... -tags=integration```

assets/overview.png

238 KB
Loading

0 commit comments

Comments
 (0)