Re: [PATCH v10 14/21] jobs: protect job.aio_context with BQL and job_mutex

[Emanuele Giuseppe Esposito]

Am 05/08/2022 um 11:12 schrieb Kevin Wolf:
> Am 25.07.2022 um 09:38 hat Emanuele Giuseppe Esposito geschrieben:
>> In order to make it thread safe, implement a "fake rwlock",
>> where we allow reads under BQL *or* job_mutex held, but
>> writes only under BQL *and* job_mutex.
> 
> Oh, so the "or BQL" part is only for job.aio_context? Okay.
> 
>> The only write we have is in child_job_set_aio_ctx, which always
>> happens under drain (so the job is paused).
>> For this reason, introduce job_set_aio_context and make sure that
>> the context is set under BQL, job_mutex and drain.
>> Also make sure all other places where the aiocontext is read
>> are protected.
>>
>> Note: at this stage, job_{lock/unlock} and job lock guard macros
>> are *nop*.
>>
>> Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
>> Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
>> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
>> ---
>>  block/replication.c |  6 ++++--
>>  blockjob.c          |  3 ++-
>>  include/qemu/job.h  | 19 ++++++++++++++++++-
>>  job.c               | 12 ++++++++++++
>>  4 files changed, 36 insertions(+), 4 deletions(-)
>>
>> diff --git a/block/replication.c b/block/replication.c
>> index 55c8f894aa..2189863df1 100644
>> --- a/block/replication.c
>> +++ b/block/replication.c
>> @@ -148,8 +148,10 @@ static void replication_close(BlockDriverState *bs)
>>      }
>>      if (s->stage == BLOCK_REPLICATION_FAILOVER) {
>>          commit_job = &s->commit_job->job;
>> -        assert(commit_job->aio_context == qemu_get_current_aio_context());
>> -        job_cancel_sync(commit_job, false);
>> +        WITH_JOB_LOCK_GUARD() {
>> +            assert(commit_job->aio_context == 
>> qemu_get_current_aio_context());
>> +            job_cancel_sync_locked(commit_job, false);
>> +        }
>>      }
> 
> .bdrv_close runs under the BQL, so why is this needed? Maybe a
> GLOBAL_STATE_CODE() annotation would be helpful, though.

I think I left it because it would be confusing to leave a _locked
function without the job lock. I'll add the GLOBAL_STATE_CODE anyways.

> 
>>      if (s->mode == REPLICATION_MODE_SECONDARY) {
>> diff --git a/blockjob.c b/blockjob.c
>> index 96fb9d9f73..9ff2727025 100644
>> --- a/blockjob.c
>> +++ b/blockjob.c
>> @@ -162,12 +162,13 @@ static void child_job_set_aio_ctx(BdrvChild *c, 
>> AioContext *ctx,
>>          bdrv_set_aio_context_ignore(sibling->bs, ctx, ignore);
>>      }
>>  
>> -    job->job.aio_context = ctx;
>> +    job_set_aio_context(&job->job, ctx);
>>  }
>>  
>>  static AioContext *child_job_get_parent_aio_context(BdrvChild *c)
>>  {
>>      BlockJob *job = c->opaque;
>> +    assert(qemu_in_main_thread());
> 
> Any reason not to use GLOBAL_STATE_CODE()?

4 months ago GLOBAL_STATE_CODE did not exist yet, and I didn't think
about updating it :)
> 
>>      return job->job.aio_context;
>>  }

>> +    /* protect against read in job_do_yield_locked */
>> +    JOB_LOCK_GUARD();
>> +    /* ensure the coroutine is quiescent while the AioContext is changed */
>> +    assert(job->pause_count > 0);
> 
> job->pause_count only shows that pausing was requested. The coroutine is
> only really quiescent if job->busy == false, too.
> 
> Or maybe job->paused is actually the one you want here.
I think job->paused works too.

Emanuele