[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Guidelines for pre-trained ML model weight binaries (Was re: Where s
Re: Guidelines for pre-trained ML model weight binaries (Was re: Where should we put machine learning model parameters?)
Fri, 07 Apr 2023 11:42:02 +0200
On ven., 07 avril 2023 at 00:50, Nathan Dehnel <firstname.lastname@example.org> wrote:
> I am uncomfortable with including ML models without their training
> data available. It is possible to hide backdoors in them.
Thanks for pointing this article! And some non-mathematical part of the
original article  are also worth to give a look. :-)
First please note that we are somehow in the case “The Open Box”, IMHO:
But what if a company knows exactly what kind of model it wants,
and simply lacks the computational resources to train it? Such a
company would specify what network architecture and training
procedure to use, and it would examine the trained model
And yeah there is nothing new ;-) when one says that the result could be
biased by the person that produced the data. Yeah, we have to trust the
trainer as we are trusting the people who generated “biased” (*) genomic
Well, it is very interesting – and scary – to see how to theoretically
exploit “misclassify adversarial examples“ as described e.g. by .
This raises questions about “Verifiable Delegation of Learning”.
>From my point of view, the tackle of such biased weights is not via
re-learning because how to draw the line between biased weights,
mistakes on their side, mistakes on our side, etc. and it requires a
high level of expertise to complete a full re-learning. Instead, it
should come from the ML community that should standardize formal methods
for verifying that the training had not been biased, IMHO.
(*) biased genomic references, for one example among many others:
Relatedly, reports have persisted of major artifacts that arise
when identifying variants relative to GRCh38, such as an
apparent imbalance between insertions and deletions (indels)
arising from systematic mis-assemblies in GRCh38
[15–17]. Overall, these errors and omissions in GRCh38 introduce
biases in genomic analyses, particularly in centromeres,
satellites, and other complex regions.