1. What is the problem? Be very detailed.
I'm running an existing ODK Central instance on Rancher/K8s/Azure as per my write-up at Installing ODK central in microsoft azure cloud.
This setup still can't send emails through our mail server, see ticket. Other apps can use this server from inside the same K8s cluster/project, which indicates the problem l could be on the config side from ODK Central.
I need to migrate to a different K8s cluster, so my IT ops colleagues have
- copied the db
- copied the resource map (config.json/local.json)
- set the same environment variables (hostname)
- spun up the same Docker images I've built from ODK Central's front- and backend
2. What app or server are you using and on what device and operating system? Include version numbers.
Service: https://hub.docker.com/r/dbcawa/odk_service 0.6.3 (built today from master)
Nginx: https://hub.docker.com/r/dbcawa/odk_nginx 0.6.3 (built today from master)
3. What you have you tried to fix the problem?
- Api specs say "for security reasons, if login fails, we won't say what failed". Fair.
- Currently trying to stand up new Central instance from scratch with copy from old db, then with empty db. Assuming central will migrate db as needed.
4. What steps can we take to reproduce the problem?
5. Anything else we should know or have? If you have a test form or screenshots or logs, attach below.
- Logs from frontend show (for failed logins):
2019/09/19 05:27:15 [error] 12#12: *14 connect() failed (113: No route to host) while connecting to upstream, client: 172.30.20.40, server: , request: "POST /v1/sessions HTTP/1.1", upstream: "http://10.244.4.96:8383/v1/sessions", host: "odkcentral-migrated.dbca.wa.gov.au", referrer: "https://odkcentral-migrated.dbca.wa.gov.au/"
172.30.20.40 - - [19/Sep/2019:05:27:15 +0000] "POST /v1/sessions HTTP/1.1" 502 166 "https://odkcentral-migrated.dbca.wa.gov.au/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0"
2019/09/19 05:39:11 [error] 12#12: *16 connect() failed (113: No route to host) while connecting to upstream, client: 172.30.20.40, server: , request: "GET /v1/projects/ HTTP/1.1", upstream: "http://10.244.4.96:8383/v1/projects/", host: "odkcentral-migrated.dbca.wa.gov.au"
So the route to the backend seems problematic.
In a shell attached to the frontend (nginx) I can ping the backend's IP and get a reply, so there is a route!
apt install inetutils-ping
ping 10.244.4.96
reply from host (works)
On my old, working instance, we can see the backend running on 8383:
root@nginx-dd7bdcbff-dljz8:/# curl 10.42.2.17:8383
{"message":"Expected an API version (eg /v1) at the start of the request URL.","code":404.2}root@nginx-dd7bdcbff-dljz8:/#
root@service-8675c4bd4c-s8j69:/usr/odk# netstat -pln
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp6 0 0 :::8383 :::* LISTEN 44/node
Active UNIX domain sockets (only servers)
Proto RefCnt Flags Type State I-Node PID/Program name Path
unix 2 [ ACC ] STREAM LISTENING 70976 33/node /usr/odk/naught.ipc
On the new instance, nothing is running on 8383"
root@service-5d55b4bc59-qzt5h:/usr/odk# apt update
root@service-5d55b4bc59-qzt5h:/usr/odk# apt install net-tools
root@service-5d55b4bc59-qzt5h:/usr/odk# netstat -plant
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 10.244.3.242:45522 151.101.28.204:80 TIME_WAIT -
tcp 0 0 10.244.3.242:45524 151.101.28.204:80 TIME_WAIT -
tcp 0 0 10.244.3.242:45776 151.101.28.204:80 TIME_WAIT -
tcp 0 0 10.244.3.242:50108 217.196.149.55:80 TIME_WAIT -
root@service-5d55b4bc59-qzt5h:/usr/odk#
root@service-5d55b4bc59-qzt5h:/usr/odk# ./start-odk.sh
running migrations..
Unhandled rejection error: SSL connection is required. Please specify SSL options and retry.
at Connection.parseE (/usr/odk/node_modules/pg/lib/connection.js:545:11)
at Connection.parseMessage (/usr/odk/node_modules/pg/lib/connection.js:370:19)
at Socket.<anonymous> (/usr/odk/node_modules/pg/lib/connection.js:113:22)
at Socket.emit (events.js:203:13)
at addChunk (_stream_readable.js:294:12)
at readableAddChunk (_stream_readable.js:275:11)
at Socket.Readable.push (_stream_readable.js:210:10)
at TCP.onStreamRead (internal/stream_base_commons.js:166:17)
starting cron..
starting server.
So, where can I specify the SSL options between backend service and db?
We found this: https://stackoverflow.com/questions/22301722/ssl-for-postgresql-connection-nodejs