Skip to content

Commit 43becde

Browse files
authored
Fix bug 1473330 - Translate.Next architecture (#989)
* Add React frontend for Translate app. This has been set up using create-react-app. * Integrate new Translate app into django. This provides a working prod and dev environment. In production, django will serve the index.html file as a template, and files built by webpack will be collected and distributed with other django static files. In development, all requests are proxied to the dev webpack server, allowing all dev niceties to be used. * Add support for websockets. * Add a bit of documentation specific to our use case. * Add redux for state management. * Show a very basic list of entities. * Enable absolute path import. * Add links to various resources. * Rearchitecture code into modules. All features should be self-contained into a module in src/modules/. All code that is shared amongst several modules should go into a module in src/core/. * Remove unused assets. * Add some architecture documentation. * Add Flow to current code for type checking. * Add unit testing with jest, enzyme and sinon. * Extend documentation, talk about type checking, modules, and list tools we use. * Move CSS rules closer to actual component. * Add django-waffle and hide translate.next behind a switch. * Better structure for the README file. * Build and test frontend in travis. * Of course it is better if dependencies are installed. * Use pushd to run commands in frontend. * Configure eslint and fix errors. * Add more code comments. * Improve documentation around local dev and Flow. * Add tests for the Translate view. * Improve documentation. * Fix serving static files for development. * Integrate Translate.Next with out docker setup. * Remove debug. * Use DEV instead of DEBUG. * Much review. Many comments. Wow better! * Flatter module public interfaces. * Import correctly from modules. * Fix docker webapp run.
1 parent 77d1b6d commit 43becde

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

43 files changed

+8619
-24
lines changed

.eslintignore

+4
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
**/*.bundle.js
22
**/*.js.map
33
**/dist/**
4+
**/build/**
45
vendor/**
56
webpack.config.js
67
coverage/**
@@ -10,4 +11,7 @@ static/*
1011
**/app/error_pages/**/*.js
1112
**/*blockrain*js
1213
assets/*
14+
**/node_modules/**
15+
docs/
16+
1317
pontoon/base/templates/js/pontoon.js

.eslintrc.js

+1
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ module.exports = {
66
env: {
77
es6: true,
88
browser: true,
9+
jest: true,
910
},
1011
parser: "babel-eslint",
1112
parserOptions: {

.gitignore

+1-1
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ tmp/*
1818
.vagrant
1919
/env/
2020
/static/
21-
.env
21+
/.env
2222
docs/_build
2323
venv
2424
/media/

.travis.yml

+3
Original file line numberDiff line numberDiff line change
@@ -32,12 +32,15 @@ install:
3232
- nvm install node
3333
- nvm use node
3434
- npm install .
35+
- pushd frontend && npm install && popd
3536
script:
3637
- pylama pontoon
3738
- pylama tests
3839
- ./node_modules/.bin/webpack
40+
- pushd frontend && npm run build && popd
3941
- python manage.py collectstatic -v0 --noinput
4042
- npm test
43+
- pushd frontend && npm test && popd
4144
- py.test --cov-append --cov-report=term --cov=. -v --create-db --migrations
4245
- ./node_modules/.bin/eslint .
4346
- codecov

Makefile

+17-1
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,22 @@ clean:
4848
test:
4949
./docker/run_tests_in_docker.sh ${ARGS}
5050

51+
test-frontend:
52+
${DOCKER} run --rm \
53+
-v `pwd`:/app \
54+
--workdir /app/frontend \
55+
--tty \
56+
--interactive \
57+
local/pontoon yarn test
58+
59+
flow:
60+
${DOCKER} run --rm \
61+
-v `pwd`:/app \
62+
-e SHELL=bash \
63+
--workdir /app/frontend \
64+
--tty --interactive \
65+
local/pontoon yarn flow:dev
66+
5167
shell:
5268
./docker/run_tests_in_docker.sh --shell
5369

@@ -65,7 +81,7 @@ build-frontend-w: assets
6581
-v `pwd`:/app \
6682
--workdir /app \
6783
-e LOCAL_USER_ID=$UID \
68-
--tty \
84+
--tty --interactive \
6985
local/pontoon npm run build-w
7086

7187
# Old targets for backwards compatibility.

docker-compose.yml

+1
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ services:
2727
command: ["/app/docker/run_webapp.sh"]
2828
ports:
2929
- "8000:8000"
30+
- "3000:3000"
3031
volumes:
3132
- .:/app
3233
entrypoint:

docker/Dockerfile

+38-18
Original file line numberDiff line numberDiff line change
@@ -13,35 +13,52 @@ RUN useradd --shell /bin/bash -c "" -m app
1313
COPY requirements.txt /app/requirements.txt
1414
COPY requirements-dev.txt /app/requirements-dev.txt
1515
COPY requirements-test.txt /app/requirements-test.txt
16-
RUN pip install -U 'pip>=8'
17-
RUN pip install --no-cache-dir --require-hashes -r requirements-dev.txt
18-
RUN pip install --no-cache-dir --require-hashes -r requirements-test.txt
16+
RUN pip install -U 'pip>=8' && \
17+
pip install --no-cache-dir --require-hashes -r requirements-dev.txt && \
18+
pip install --no-cache-dir --require-hashes -r requirements-test.txt
1919

20-
# Install nodejs and npm from Nodesource's 8.x branch
21-
# RUN DEBIAN_FRONTEND=noninteractive apt-get install -y curl
20+
# Install nodejs and npm from Nodesource's 8.x branch, as well as yarn
2221
RUN curl -s https://deb.nodesource.com/gpgkey/nodesource.gpg.key | apt-key add - && \
2322
echo 'deb https://deb.nodesource.com/node_9.x jessie main' > /etc/apt/sources.list.d/nodesource.list && \
2423
echo 'deb-src https://deb.nodesource.com/node_9.x jessie main' >> /etc/apt/sources.list.d/nodesource.list
25-
RUN DEBIAN_FRONTEND=noninteractive apt-get update
26-
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y nodejs
24+
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - && \
25+
echo 'deb https://dl.yarnpkg.com/debian/ stable main' > /etc/apt/sources.list.d/yarn.list
26+
RUN DEBIAN_FRONTEND=noninteractive apt-get update && \
27+
DEBIAN_FRONTEND=noninteractive apt-get install -y nodejs yarn
2728

2829
# Create some folders and give them to the app user
29-
RUN mkdir -p /webapp-frontend-deps && chown app:app /webapp-frontend-deps
30+
RUN mkdir -p /webapp-frontend-deps && \
31+
chown app:app /webapp-frontend-deps
32+
3033
RUN chown app:app /app
3134

3235
# Create the folder for front-end assets
33-
RUN mkdir -p /webapp-frontend-deps/assets && chown app:app /webapp-frontend-deps/assets
36+
RUN mkdir -p /webapp-frontend-deps/assets && \
37+
chown app:app /webapp-frontend-deps/assets
38+
39+
# Create the folders for Translate.Next
40+
RUN mkdir -p /app/frontend && \
41+
mkdir -p /webapp-frontend-deps/frontend/build
3442

35-
# Link to the node modules installed with npm. We install those in a different folder because
36-
# when using this container in development, we replace the /app folder with a volume. By putting
37-
# those dependencies outside of /app, we can reuse them more easily.
38-
RUN ln -s /webapp-frontend-deps/node_modules /app/node_modules
39-
RUN ln -s /webapp-frontend-deps/assets /app/assets
43+
RUN chown -R app:app /webapp-frontend-deps
4044

4145
# Install node requirements
4246
COPY ./package.json /webapp-frontend-deps/package.json
4347
COPY ./package-lock.json /webapp-frontend-deps/package-lock.json
44-
RUN cd /webapp-frontend-deps/ && npm install
48+
RUN cd /webapp-frontend-deps && npm install
49+
50+
COPY ./frontend/package.json /webapp-frontend-deps/frontend/package.json
51+
COPY ./frontend/yarn.lock /webapp-frontend-deps/frontend/yarn.lock
52+
RUN cd /webapp-frontend-deps/frontend && yarn install
53+
54+
# Link to the node modules installed with npm and yarn. We install those in
55+
# different folders because when using this container in development,
56+
# we replace the /app folder with a volume. By putting those dependencies
57+
# outside of /app, we can reuse them more easily.
58+
RUN ln -s /webapp-frontend-deps/node_modules /app/node_modules && \
59+
ln -s /webapp-frontend-deps/assets /app/assets && \
60+
ln -s /webapp-frontend-deps/frontend/node_modules /app/frontend/node_modules && \
61+
ln -s /webapp-frontend-deps/frontend/build /app/frontend/build
4562

4663
COPY . /app/
4764
COPY ./docker/config/webapp.env /app/.env
@@ -55,11 +72,14 @@ ENV PYTHONPATH /app
5572
ENV WEBPACK_BINARY /app/node_modules/.bin/webpack
5673
ENV YUGLIFY_BINARY /app/node_modules/.bin/yuglify
5774

75+
# Run webpack to compile JS files
76+
RUN cd /app/ && $WEBPACK_BINARY
77+
78+
# Build Translate.Next frontend resources
79+
RUN cd /app/frontend/ && yarn build
80+
5881
# Run collectstatic in container which puts files in the default place for
5982
# static files.
6083
RUN cd /app/ && python manage.py collectstatic --noinput
6184

62-
# Run webpack to compile JS files
63-
RUN cd /app/ && $WEBPACK_BINARY
64-
6585
CMD ["/app/docker/run_webapp.sh"]

docker/entrypoint_webapp.sh

+3-1
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,9 @@
11
#!/bin/bash
22

3-
# Make sure a link to the node modules is created.
3+
# Make sure links to the node modules are created.
44
ln -s /webapp-frontend-deps/node_modules /app/node_modules
5+
ln -s /webapp-frontend-deps/frontend/node_modules /app/frontend/node_modules
6+
ln -s /webapp-frontend-deps/frontend/build /app/frontend/build
57
ln -s /webapp-frontend-deps/assets /app/assets
68

79
# Add local user

docker/run_webapp.sh

+4-1
Original file line numberDiff line numberDiff line change
@@ -8,5 +8,8 @@ git rev-parse HEAD > static/revision.txt
88
echo ">>> Setting up the db for Django"
99
python manage.py migrate
1010

11+
echo ">>> Starting frontend build process in the background"
12+
cd frontend && yarn start &
13+
1114
echo ">>> Starting local server"
12-
python manage.py runserver 0.0.0.0:8000
15+
python manage.py runserver --nostatic 0.0.0.0:8000

frontend/.env

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
NODE_PATH = 'src/'

frontend/.flowconfig

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
[ignore]
2+
3+
[include]
4+
5+
[libs]
6+
7+
[lints]
8+
9+
[options]
10+
module.system=node
11+
module.system.node.resolve_dirname=node_modules
12+
module.system.node.resolve_dirname=./src
13+
14+
[strict]

frontend/.gitignore

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# See https://help.github.com/ignore-files/ for more about ignoring files.
2+
3+
# dependencies
4+
/node_modules
5+
6+
# testing
7+
/coverage
8+
9+
# production
10+
/build
11+
12+
# misc
13+
.DS_Store
14+
.env.local
15+
.env.development.local
16+
.env.test.local
17+
.env.production.local
18+
19+
npm-debug.log*
20+
yarn-debug.log*
21+
yarn-error.log*

frontend/README.md

+162
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,162 @@
1+
# Translate.Next — a better Translate app for Pontoon
2+
3+
4+
## Tools
5+
6+
<table>
7+
<tr>
8+
<td>Bootstrapper</td>
9+
<td>create-react-app — https://github.com/facebook/create-react-app</td>
10+
</tr>
11+
<tr>
12+
<td>Package manager</td>
13+
<td>Yarn — https://yarnpkg.com/</td>
14+
</tr>
15+
<tr>
16+
<td>Views</td>
17+
<td>React — https://reactjs.org/</td>
18+
</tr>
19+
<tr>
20+
<td>Data</td>
21+
<td>Redux — https://redux.js.org/</td>
22+
</tr>
23+
<tr>
24+
<td>Tests</td>
25+
<td>Jest — http://jestjs.io/</td>
26+
</tr>
27+
<tr>
28+
<td>Type checking</td>
29+
<td>Flow — https://flow.org/</td>
30+
</tr>
31+
<tr>
32+
<td>Localization</td>
33+
<td>Fluent — https://projectfluent.org/</td>
34+
</tr>
35+
<tr>
36+
<td>Style</td>
37+
<td>CSS</td>
38+
</tr>
39+
</table>
40+
41+
42+
## Code architecture
43+
44+
### Where code goes
45+
46+
`src/core/` contains features that are shared in the application, in the form of modules. There should be as little code as possible in this folder.
47+
48+
`src/modules/` contains the self-contained features of the application, separated in modules.
49+
50+
`src/rootReducer.js` creates the main reducer to be used with Redux. When adding a new module with a reducer, make sure to include that reducer to `rootReducer`.
51+
52+
### Modules
53+
54+
Each module should be in a folder and have an `index.js` file that serves as the module's public interface. Outside of the module, always import from the module's index, and never from specific files. This allows us to easily evolve modules and keep things decoupled, which reduces code complexity.
55+
56+
Here are the files commonly found in a module:
57+
58+
- `index.js` — public interface of the module
59+
- `actions.js` — actions that can be used to fetch data from an API, trigger changes, etc.
60+
- `reducer.js` — a single Redux reducer (if you need to have more than one reducer, you probably actually need to make several modules)
61+
- `constants.js` — a list of constants required by the module or other modules. It is recommended to define a NAME constant here which should be a unique identifier of the module. This name will be used in `combineReducers` and in all `mapStateToProps` to identify the subtree of the global state relevant to this module.
62+
- `components/` — a folder containing components, with file names in CamelCase
63+
64+
Of course, more can be added if needed. For example, modules with a high number of action types might want to have an `actionTypes.js` file to separate them from actions.
65+
66+
67+
## Running and deploying
68+
69+
### This feature is behind a Switch
70+
71+
While this is under development, the feature is hidden behing a feature switch, and thus is not accessible by default. In order to turn it on, you have to run `./manage.py waffle_switch translate_next on --create`, then restart your web server. To turn it off, run `./manage.py waffle_switch translate_next off`.
72+
73+
### Production
74+
75+
The only required step for the front-end is to build static files with `yarn build`. Django is configured to collect the `index.html` and static files from the `build` folder and put them with other static files. All of that is automated for deployement to Heroku.
76+
77+
### Development
78+
79+
If you're using docker, `make run` automatically starts both a webpack server (on port 3000) and a Django server (on port 8000). Django is the server you want to hit, and it will then proxy appropriate requests to the webpack server.
80+
81+
A common case during development is to have 3 terminals open: one for the dev servers, one for the tests and one for Flow:
82+
83+
# terminal 1
84+
$ make run
85+
86+
# terminal 2
87+
$ make test-frontend
88+
89+
# terminal 3
90+
$ make flow
91+
92+
#### Enabling websocket and warm-reloading for dev
93+
94+
Currently websocket requests are redirected by Django to the webpack server. Sadly, by default major browsers do not support websocket redirection. To enable it in Firefox, go to `about:config` and turn `network.websocket.auto-follow-http-redirect` to `true`. Note that there is a bug filed to get rid of that option entirely: https://bugzilla.mozilla.org/show_bug.cgi?id=1052909
95+
96+
As far as we know, it is not possible to make that work in Chrome or Edge. This only impacts development, as there's no hot reloading in production.
97+
98+
If you can't turn on websockets, you will see errors in the console (that's not very impacting) and you'll have to reload your Django server regularly, because polling requests don't close, and after so many web page reloads, the Django process won't be able to accept new requests.
99+
100+
101+
## Type checking
102+
103+
Our code uses Flow to enforce type checking. This is a good way to significantly improve resilience to bugs, and it removes some burden from unit tests (because Flow ensures that we use functions and components correctly).
104+
105+
To check for Flow issues during development while you edit files, run:
106+
107+
yarn flow:dev
108+
109+
To learn more, you can read [Why use static types in JavaScript?](https://medium.freecodecamp.org/why-use-static-types-in-javascript-part-1-8382da1e0adb) or the official [Flow documentation](https://flow.org/en/docs/). Additionally, you can read through the [web-ext guide](https://github.com/mozilla/web-ext/blob/master/CONTRIBUTING.md#check-for-flow-errors) for hints on how to solve common Flow errors.
110+
111+
Until we define our set of rules, please refer to the [addons team's Flow manifesto](https://github.com/mozilla/addons-frontend/#flow) regarding specific usage and edge cases.
112+
113+
114+
## Testing
115+
116+
Tests are run using [`jest`](https://facebook.github.io/jest/). We use [`enzyme`](http://airbnb.io/enzyme/docs/api/) for mounting React components and [`sinon`](http://sinonjs.org/) for mocking.
117+
118+
To run the test suite, use:
119+
120+
$ yarn test
121+
122+
It will start an auto-reloading test runner, that will refresh every time you make a change to the code or tests.
123+
124+
Tests are put in files called `fileToTest.test.js` in the same directory as the file to test. Inside test files, test suites are created. There should be one test suite per component, using this notation:
125+
126+
```javascript
127+
describe('<Component>', () => {
128+
// test suite here
129+
});
130+
```
131+
132+
Individual tests follow `mocha`'s syntax:
133+
134+
```javascript
135+
it('does something', () => {
136+
// unit test here
137+
});
138+
```
139+
140+
We use `jest`'s [`expect`](https://facebook.github.io/jest/docs/en/expect.html) assertion tool.
141+
142+
143+
## Development resources
144+
145+
### Integration between Django and React
146+
147+
- [Making React and Django play well together — Fractal Ideas](https://fractalideas.com/blog/making-react-and-django-play-well-together/)
148+
- [Making React and Django play well together - the “hybrid app” model — Fractal Ideas](https://fractalideas.com/blog/making-react-and-django-play-well-together-hybrid-app-model/)
149+
150+
### Code architecture
151+
152+
- [Three Rules For Structuring (Redux) Applications — Jack Hsu](https://jaysoo.ca/2016/02/28/organizing-redux-application/)
153+
- [The Anatomy Of A React & Redux Module (Applying The Three Rules) — Jack Hsu](https://jaysoo.ca/2016/02/28/applying-code-organization-rules-to-concrete-redux-code/)
154+
- [Additional Guidelines For (Redux) Project Structure — Jack Hsu](https://jaysoo.ca/2016/12/12/additional-guidelines-for-project-structure/)
155+
156+
### Tools documentation
157+
158+
- [React](https://reactjs.org/docs/getting-started.html)
159+
- [Redux](https://redux.js.org/)
160+
- [create-react-app](https://github.com/facebook/create-react-app/blob/master/packages/react-scripts/template/README.md)
161+
- [Flow](https://flow.org/en/docs/)
162+
- [Jest](http://jestjs.io/docs/en/getting-started)

0 commit comments

Comments
 (0)