Fossil

Diff
Login

Diff

Differences From Artifact [e3ef13f6a1]:

To Artifact [c4f2b6c4ae]:


153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193

194
195
196
197
198
199
200
201
202
203

```
  --volume ~/museum/my-project.fossil:/jail/museum/repo.fossil
```

That lets us have a convenient file name for the project outside the
container while letting the configuration inside the container refer to
the generic “`/museum/repo.fossil`” name. Why should we have to rename
the container generically on the outside just to placate the container?

The reason is, you might be serving that repo with [WAL mode][wal]
enabled. If you map the repo DB alone into the container, the Fossil
instance inside the container will write the `-journal` and `-wal` files
alongside the mapped-in repository inside the container.  That’s fine as
far as it goes, but if you then try using the same repo DB from outside
the container while there’s an active WAL, the Fossil instance outside
won’t know about it. It will think it needs to write *its own*
`-journal` and `-wal` files *outside* the container, creating a high
risk of [database corruption][dbcorr].

If we map a whole directory, both sides see the same set of WAL files,
so there is at least a *hope* that WAL will work properly across that
boundary. The success of the scheme depends on the `mmap()` and shared
memory system calls being coordinated properly by the OS kernel the two
worlds share. 

At some point, someone should perform tests in the hopes of *failing* to
create database corruption in this scenario.

Why the tortured grammar? Because you cannot prove a negative, being in
this case “SQLite will not corrupt the database in WAL mode if there’s a
container barrier in the way.” All you can prove is that a given test
didn’t cause corruption. With enough tests of sufficient power, you can
begin to make definitive statements, but even then, science is always
provisional, awaiting a single disproving experiment. Atop that, OCI
container runtimes give the sysadmin freedom to impose barriers between
the two worlds, so even if you convince yourself that WAL mode is safe
in a given setup, it’s possible to configure it to fail. As if that
weren’t enough, different container runtimes have different defaults,
including details like whether shared memory is truly shared between
the host and its containers.


Until someone gets around to establishing this ground truth and scoping
its applicable range, my advice to those who want to use WAL mode on
containerized servers is to map the whole directory as shown in these
examples, but then isolate the two sides with a secondary clone. On the
outside, you say something like this:

```
  $ fossil clone https://user@example.com/myproject ~/museum/myproject.fossil
```







|
|

















<
<
|
<
<
<
<
<
|
<
<
<
<
<
|
>

<
|







153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178


179





180





181
182
183

184
185
186
187
188
189
190
191

```
  --volume ~/museum/my-project.fossil:/jail/museum/repo.fossil
```

That lets us have a convenient file name for the project outside the
container while letting the configuration inside the container refer to
the generic “`/museum/repo.fossil`” name. Why should we have to name
the repo generically on the outside merely to placate the container?

The reason is, you might be serving that repo with [WAL mode][wal]
enabled. If you map the repo DB alone into the container, the Fossil
instance inside the container will write the `-journal` and `-wal` files
alongside the mapped-in repository inside the container.  That’s fine as
far as it goes, but if you then try using the same repo DB from outside
the container while there’s an active WAL, the Fossil instance outside
won’t know about it. It will think it needs to write *its own*
`-journal` and `-wal` files *outside* the container, creating a high
risk of [database corruption][dbcorr].

If we map a whole directory, both sides see the same set of WAL files,
so there is at least a *hope* that WAL will work properly across that
boundary. The success of the scheme depends on the `mmap()` and shared
memory system calls being coordinated properly by the OS kernel the two
worlds share. 



There is [a plan](https://tangentsoft.com/sqlite/dir/walbanger?ci=trunk)





for proving to a reasonable level of confidence that using WAL across a





container boundary is safe, but this effort is still in the early stages
as of this writing.


Until that’s settled, my advice to those who want to use WAL mode on
containerized servers is to map the whole directory as shown in these
examples, but then isolate the two sides with a secondary clone. On the
outside, you say something like this:

```
  $ fossil clone https://user@example.com/myproject ~/museum/myproject.fossil
```