-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathdata-migration-steps.yml
176 lines (142 loc) · 8.55 KB
/
data-migration-steps.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
using this branch:
- 1 copy the database dump.sql backup file to:
sql/mysql
- 2 rename the dump to openmrs.sql:
- 3 spin-up mysql and openmrs services only:
docker compose up mysql openmrs --build
- 4 access openmrs at:
localhost/openmrs
- 5 login as:
admin:Admin123
- 6 proceed with the liquibase migrations:
by clicking the forward button at the bottom of the page
- 7 wait for all the liquibase migrations to go through:
- 8 access localhost/openmrs which should display:
OpenMRS Platform 2.* Running!
- 9 terminate the docker containers and repeat steps 3 to 7 for versions of OpenMRS Core; 2.2.1, 2.3.6, 2.4.5 and 2.5.12:
- note: do not terminate the containers when running with 2.5.12 as the corresonding database will have to be backup in the next step
- note: to change the openmrs version, access ./openmrs/Dockerfile and update OPENMRS_WAR_URL accordingly
- 10 in a different terminal backup the updated opnemrs database:
follow instructions in the "Backup databases and filestores" section in the readme (only run the openmrs-db-backup service)
- 11 stop the containers by terminating the previous command (docker compose up mysql openmrs --build) under the termin in which it's running:
(control + C on the keyboard)
- 12 Using the backup openmrs dump, restore it into Ozone appropriately and start only the mysql instance:
- 13 run the following queries in preparations to start the acompaning openmrs service:
UPDATE concept_reference_source SET hl7_code = 'SCT-legacy' WHERE hl7_code = 'SCT' AND uuid like '03f574a2-dce4-11ec-925e-0242ac160002';
UPDATE concept SET uuid = CONCAT(uuid, REPEAT('A', 36 - LENGTH(uuid))) WHERE uuid IN ('5090AAAAAAAAAAAAAAAAAAAAAAAAAAAA', '5089AAAAAAAAAAAAAAAAAAAAAAAAAAAA');
UPDATE drug_order SET dosing_type = 'org.openmrs.SimpleDosingInstructions' WHERE dosing_type = 'org.openmrs.module.bahmniemrapi.drugorder.dosinginstructions.FlexibleDosingInstructions';
INSERT INTO test_order (order_id) SELECT o.order_id FROM orders o JOIN order_type ot ON o.order_type_id = ot.order_type_id WHERE ot.uuid = 'f8ae333e-9d1a-423e-a6a8-c3a0687ebcf2';
UPDATE orders o set o.order_type_id=(select order_type_id from order_type where uuid like '52a447d3-a64a-11e3-9aeb-50e549534c5e' limit 1) WHERE o.order_type_id = (select order_type_id from order_type where uuid like 'f8ae333e-9d1a-423e-a6a8-c3a0687ebcf2' limit 1);
CREATE TABLE openmrs.patientflags_tag_role (
tag_id INT NOT NULL,
role VARCHAR(50) NOT NULL,
CONSTRAINT patientflags_tag_role_ibfk_2 FOREIGN KEY (role) REFERENCES openmrs.role(role)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_general_ci;
## the above tryes to avoid the following error when patientflags module liquibase migrations are run : ERROR 1005 (HY000): Can't create table `openmrs`.`patientflags_tag_role` (errno: 150 "Foreign key constraint is incorrectly formed")
- 14 start the rest of the containers in addition to the already running mysql containers:
- 15 restart all the Ozone containers and where neccessary it might be required to run the query on the openmrs database:
UPDATE liquibasechangeloglock SET LOCKED = '0', LOCKEDBY = NULL, LOCKGRANTED = NULL WHERE ID = 1;
- 16 access the running ozone based openmrs instance and rebuild the search index:
- 17 migrate encounter diagnoses from obs to their more specialised data model:
retire the following duplicate OCL concepts (name:Diagnosis or problem, non-coded, uuid:161602AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA), (name:Problem list, uuid:1284AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA)
via legacy ui, navigate to 'Administration > Migrate Existing Diagnosis Data' and proceed to migrate diagnoses, this will take a long while
query the encounter_diagnosis table to ensure completion by getting a consistent result for 1 minute using this query - `select count(*) from encounter_diagnosis;`
- 18 run following query to generate csv file that maps file on the file system to the corresponding obs:
docker exec -it ozone-hsc-mysql-1 rm /tmp/migration_file_paths.csv # if exists
docker exec -it ozone-hsc-mysql-1 mysql -uroot -pmysql_root_password openmrs -e "SELECT 'file_path', 'new_file_path'
UNION ALL
SELECT
o.value_text AS file_path,
CONCAT(
REPLACE(SUBSTRING_INDEX(o.value_text, '/', 1), '/', '-'),
'-',
REPLACE(REPLACE(SUBSTRING_INDEX(SUBSTRING_INDEX(o.value_text, '/', -1), '.', 1), ' ', '-'), '/', '-'),
'_', o.uuid,
'.', SUBSTRING_INDEX(o.value_text, '.', -1)
) AS new_file_path
FROM obs o
JOIN concept c ON o.concept_id = c.concept_id
WHERE c.uuid LIKE '81f330b5-3f10-11e4-adec-0800271c1b75'
AND o.value_text IS NOT NULL
INTO OUTFILE '/tmp/migration_file_paths.csv'
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n';"
docker cp ozone-hsc-mysql-1:/tmp/migration_file_paths.csv migration_file_paths.csv
- 19 Run following bash script as follows:
touch complex_obs_migration_script.sh
nano complex_obs_migration_script.sh
##########################
##### Copy over the following script and save:
#!/bin/bash
# Get the directory where the script is located
script_dir="$(dirname "$(realpath "$0")")"
# Set the CSV file name (ensure this file exists and is in the same directory as the script)
csv_file="$script_dir/migration_file_paths.csv"
# Check if the CSV file exists
if [[ ! -f "$csv_file" ]]; then
echo "CSV file not found!"
exit 1
fi
# Read the CSV file line by line
while IFS=',' read -r file_path new_file_path; do
# Skip the header row
if [[ "$file_path" == "file_path" ]]; then
continue
fi
# Make file paths relative to the script's directory
abs_file_path="$script_dir/$file_path"
abs_new_file_path="/home/ubuntu/complex_obs/$new_file_path"
# Check if the source file exists
if [[ ! -f "$abs_file_path" ]]; then
echo "File not found: $abs_file_path"
continue
fi
# Create the target directory if it does not exist
target_dir=$(dirname "$abs_new_file_path")
if [[ ! -d "$target_dir" ]]; then
echo "Target directory does not exist, creating: $target_dir"
mkdir -p "$target_dir"
fi
# Move the file
mv "$abs_file_path" "$abs_new_file_path"
if [[ $? -eq 0 ]]; then
echo "Moved: $abs_file_path to $abs_new_file_path"
else
echo "Failed to move: $abs_file_path"
fi
done < "$csv_file"
echo "File move process completed."
# save script and run it from the root directory containing the un migrated files
chmod +x complex_obs_migration_script.sh
./complex_obs_migration_script.sh
- 21 copy copy the moved files/documents into the complex obs directory inside the OpenMRS container:
docker cp /home/ubuntu/complex_obs/. oz-hsc-403-openmrs-1:/openmrs/data/complex_obs/
- 22 run the following database migration turning the migrated obs on the file system into complex obs:
docker exec -it ozone-hsc-mysql-1 mysql -uroot -pmysql_root_password openmrs -e "UPDATE obs o
JOIN concept c ON o.concept_id = c.concept_id
SET
o.value_complex = CONCAT(
'm3ks | instructions.default | ',
CASE
WHEN SUBSTRING_INDEX(o.value_text, '.', -1) IN ('jpeg', 'png')
THEN CONCAT('image/', SUBSTRING_INDEX(o.value_text, '.', -1))
ELSE CONCAT('application/', SUBSTRING_INDEX(o.value_text, '.', -1))
END,
' | ',
CONCAT(
REPLACE(SUBSTRING_INDEX(o.value_text, '/', 1), '/', '-'),
'-',
REPLACE(REPLACE(SUBSTRING_INDEX(SUBSTRING_INDEX(o.value_text, '/', -1), '.', 1), ' ', '-'), '/', '-'),
'_', o.uuid,
'.', SUBSTRING_INDEX(o.value_text, '.', -1)
)
),
o.concept_id = (SELECT concept_id FROM concept WHERE uuid =
CASE
WHEN SUBSTRING_INDEX(o.value_text, '.', -1) IN ('jpeg', 'png')
THEN '7cac8397-53cd-4f00-a6fe-028e8d743f8e'
ELSE '42ed45fd-f3f6-44b6-bfc2-8bde1bb41e00'
END LIMIT 1),
o.value_text = NULL
WHERE c.uuid LIKE '81f330b5-3f10-11e4-adec-0800271c1b75'
AND o.value_text IS NOT NULL;"