[tycho-user] Nexus p2 unzip plugin: Read lock during ZIP transfer to proxy

[Irene Wang]

Hello,

Hopefully, this is the correct list to ask about an issue with the Nexus
p2 unzip plugin. We're using version 0.15.0-SNAPSHOT of the p2 unzip
plugin with instances of Nexus Pro 2.14.2-01. We are having problems
with blocked read access to a ZIP artifact (and its corresponding p2
unzip virtual path) while that ZIP artifact is being transferred to a
regular Maven proxy repository. This blocked read issue does not occur
when there is no p2 unzip virtual repository sourcing the repository.

On our main Nexus instance, we have a Maven repository (A) with a p2
virtual repository (A') that sources repository A.
On another "mirror" Nexus server, we have a Maven proxy repository (B)
of the same repository A.

The symptom is that whenever a ZIP artifact is published to A and the
proxy repository B has requested that ZIP artifact before the p2
unzipped version on A' is requested, both the original artifact on A and
and the virtual artifact on A' are unreadable until the replication of
the ZIP from A to B is complete (keeps attempting to load until the
request times out). Once the transfer of the ZIP to the proxy repository
B is finished, all paths on A, A' and B become accessible. The expected
behaviour is that A and A' on the main instance would still be readable
during the transfer of the ZIP to the proxy repository B. Again, this
locking only happens when a p2 unzip virtual repository A' is configured
with its source as A.

As our "mirror" Nexus is located in another geographical region, the
transfer of ZIP artifacts from A to B can take quite some time, and in
the meantime, the corresponding artifacts cannot be requested on the
main Nexus, whether in ZIP (A) or p2 unzip (A') format.

Would anyone have any insight into a locking mechanism specific to the
p2 unzip plugin that could explain what we're seeing, and if so, would
there be a workaround or fix?

Thanks very much in advance,

Irene

[Irene Wang]

An update after some investigation of thread dumps and the p2 unzip
plugin code.

The waits happen here, waiting for
org.eclipse.tycho.nexus.internal.plugin.cache.PathLock#PathLockMonitor:
https://github.com/eclipse/tycho.nexus/blob/master/unzip-repository-plugin/src/main/java/org/eclipse/tycho/nexus/internal/plugin/cache/UnzipCache.java#L70

The comment in the code states that it's a workaround for
https://issues.sonatype.org/browse/NEXUS-3622, which was resolved in Nov
2011.
Is there any chance that this code conflicts with later versions of Nexus?

Additional observation:
- Any HTTP get request in progress of the ZIP artifact (downloading via
browser) makes the p2 unzip plugin's copy of the ZIP from the source
repository tree to the virtual repository tree wait, so it is not
specific to proxy repositories in any way.

I have attached the thread dump taken during the read lock, the
interesting lines are 16 and 17.

Thanks again.

Post by Irene Wang
Hello,
Hopefully, this is the correct list to ask about an issue with the
Nexus p2 unzip plugin. We're using version 0.15.0-SNAPSHOT of the p2
unzip plugin with instances of Nexus Pro 2.14.2-01. We are having
problems with blocked read access to a ZIP artifact (and its
corresponding p2 unzip virtual path) while that ZIP artifact is being
transferred to a regular Maven proxy repository. This blocked read
issue does not occur when there is no p2 unzip virtual repository
sourcing the repository.
On our main Nexus instance, we have a Maven repository (A) with a p2
virtual repository (A') that sources repository A.
On another "mirror" Nexus server, we have a Maven proxy repository (B)
of the same repository A.
The symptom is that whenever a ZIP artifact is published to A and the
proxy repository B has requested that ZIP artifact before the p2
unzipped version on A' is requested, both the original artifact on A
and and the virtual artifact on A' are unreadable until the
replication of the ZIP from A to B is complete (keeps attempting to
load until the request times out). Once the transfer of the ZIP to the
proxy repository B is finished, all paths on A, A' and B become
accessible. The expected behaviour is that A and A' on the main
instance would still be readable during the transfer of the ZIP to the
proxy repository B. Again, this locking only happens when a p2 unzip
virtual repository A' is configured with its source as A.
As our "mirror" Nexus is located in another geographical region, the
transfer of ZIP artifacts from A to B can take quite some time, and in
the meantime, the corresponding artifacts cannot be requested on the
main Nexus, whether in ZIP (A) or p2 unzip (A') format.
Would anyone have any insight into a locking mechanism specific to the
p2 unzip plugin that could explain what we're seeing, and if so, would
there be a workaround or fix?
Thanks very much in advance,
Irene