Discussion:
Tycho 0.16 + Jenkins + NTLM2 proxy
(too old to reply)
Girard-reydet, Vincent
2013-02-18 15:07:10 UTC
Permalink
Hi folks,



I'm struggling to configure a Maven build using Tycho on Jenkins behind
my company's proxy. I have a set of modules that build perfectly well
from the console (I just rebuilt everything using a fresh m2 repo and
the build is completely successful). Now I' trying to automate it with
Jenkins (see the full env spec at the end of the email). I have
configured a Maven build in Jenkins. For the local and global settings,
I'm pointing at the exact same settings file I'm using in the console.
This settings file contains the proxy definition, with the username,
password and everything (and working fine in console). The build is
using a private Maven repo.

For my build, I'm using a target platform file (NOT using p2
repositories in the pom.xml), as per the below. This platform is
resolved OK when building in console.



The Jenkins build uses correctly the proxy information to download all
Maven plugins (including Tycho plugins). However, when Tycho starts
resolving the platform, it somehow fails to collect the proxy
information (which it should since TYCHO-279 is now closed). I tried all
the workarounds described to circumvene TYCHO-279 (using the ECF proxy
properties), I tried also using a Nexus repo, nothing worked. I'm a bit
screwed new, it seems to be somehow Jenkins-specific but I don't' have a
clue what's going on.



Would someone be able to help me investigate the problem?



Environment spec:

- Windows 7

- NTLM2 proxy

- Maven 3.0.4

- JDK 1.6.39 x86_64

- Jenkins v1.501 (out of the box install, running as a Windows
service)

- Tycho v0.16



Platform configuration in parent pom:



<plugin>

<groupId>org.eclipse.tycho</groupId>

<artifactId>target-platform-configuration</artifactId>

<version>${tycho.version}</version>

<configuration>

<target>

<artifact>

<groupId>FAKE_GROUP_ID</groupId>

<artifactId>target</artifactId>

<version>1.0.0</version>

<classifier>indigo</classifier>

</artifact>

</target>

<environments>

<environment>

<os>win32</os>

<ws>win32</ws>

<arch>x86</arch>

</environment>

<environment>

<os>win32</os>

<ws>win32</ws>

<arch>x86_64</arch>

</environment>

</environments>

</configuration>

</plugin>



Platform definition :

<?xml version="1.0" encoding="UTF-8" standalone="no"?>

<?pde version="3.6"?>



<target name="indigo" sequenceNumber="14">

<locations>

<location includeAllPlatforms="false" includeMode="slicer"
includeSource="false" type="InstallableUnit">

<unit id="org.eclipse.equinox.server.servletbridge.feature.group"
version="1.0.100.v20110502-42DF9oB5865K5D"/>

<repository location="http://download.eclipse.org/eclipse/updates/3.7"/>

</location>

<location includeAllPlatforms="false" includeMode="slicer"
includeSource="false" type="InstallableUnit">

<unit id="org.eclipse.ajdt.feature.group"
version="2.2.0.e37x-RELEASE-20120704-0900"/>

<unit id="org.eclipse.equinox.weaving.sdk.feature.group"
version="1.0.200.I20120427-0800"/>

<repository
location="http://download.eclipse.org/tools/ajdt/37/update"/>

</location>

<location includeAllPlatforms="false" includeMode="slicer"
includeSource="false" type="InstallableUnit">

<unit id="org.eclipse.sdk.ide" version="3.7.2.M20120208-0800"/>

<unit id="org.eclipse.emf.ecore.feature.group"
version="2.7.1.v20120127-1122"/>

<unit id="org.eclipse.emf.sdk.feature.group"
version="2.7.2.v20120130-0943"/>

<unit id="org.eclipse.emf.compare.feature.group"
version="1.2.2.v20120214-0915"/>

<unit id="org.eclipse.emf.query.feature.group"
version="1.5.0.v20110502-1600-218Z7w311A142A1142"/>

<unit id="org.eclipse.draw2d.feature.group"
version="3.7.2.v20110927-2020-4617w3122212803131"/>

<unit id="org.eclipse.xsd.sdk.feature.group"
version="2.7.1.v20120130-0943"/>

<unit id="org.eclipse.objectteams.otequinox.feature.group"
version="2.0.2.201202051448"/>

<unit id="org.eclipse.equinox.sdk.feature.group"
version="3.7.1.R37x_v20110907-7M7W8i8eNV4WsRkue-4Vq4J6pCyW"/>

<unit id="javax.transaction" version="1.1.1.v201105210645"/>



<unit id="org.eclipse.equinox.executable.feature.group"
version="3.5.0.v20110530-7P7NFUFFLWUl76mart"/>

<unit id="org.eclipse.jdt.feature.group"
version="3.7.1.r371_v20110810-0800-7z8gFcoFMLfTabvKsR5Qm9rBGEBK"/>

<unit id="org.eclipse.rcp.sdk.id" version="3.7.1.M20110909-1335"/>



<repository location="http://download.eclipse.org/releases/indigo"/>

</location>

</locations>

<environment>

<arch>x86</arch>

</environment>

</target>



Error log in Jenkins (no error when building from console):



Downloaded:
http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plug
in/1.3/aspectj-maven-plugin-1.3.jar
<http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plu
gin/1.3/aspectj-maven-plugin-1.3.jar> (37 KB at 63.6 KB/sec)
[INFO] Computing target platform for MavenProject:
<MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @
C:\jenkins_workspace\test\<MODULE_ID_HIDDEN>\pom.xml
18-Feb-2013 12:30:08
org.apache.commons.httpclient.auth.AuthChallengeProcessor
selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 12:30:08 org.apache.commons.httpclient.HttpMethodDirector
processProxyAuthChallenge
INFO: No credentials available for NTLM <any realm>@iaccess.bp.com:80
mavenExecutionResult exceptions not empty
message : Internal error: java.lang.RuntimeException: Failed to resolve
target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
cause : Failed to resolve target definition
C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
Stack trace :
org.apache.maven.InternalErrorException: Internal error:
java.lang.RuntimeException: Failed to resolve target definition
C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:168)
at
org.jvnet.hudson.maven3.launcher.Maven3Launcher.main(Maven3Launcher.java
:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launche
r.java:329)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:2
39)
at
org.jvnet.hudson.maven3.agent.Maven3Main.launch(Maven3Main.java:158)
at hudson.maven.Maven3Builder.call(Maven3Builder.java:100)
at hudson.maven.Maven3Builder.call(Maven3Builder.java:66)
at hudson.remoting.UserRequest.perform(UserRequest.java:118)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:326)
at
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorS
ervice.java:72)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecuto
r.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
va:918)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.RuntimeException: Failed to resolve target
definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
at
org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileCont
entToTargetPlatform(P2TargetPlatformResolver.java:338)
at
org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.computeTargetPlat
form(P2TargetPlatformResolver.java:207)
at
org.eclipse.tycho.core.resolver.DefaultTychoDependencyResolver.resolvePr
oject(DefaultTychoDependencyResolver.java:94)
at
org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.afterProject
sRead(TychoMavenLifecycleParticipant.java:82)
at
org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:274)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
... 19 more
Caused by:
org.eclipse.tycho.p2.target.facade.TargetDefinitionResolutionException:
Failed to load metadata repository from location
http://download.eclipse.org/eclipse/updates/3.7
<http://download.eclipse.org/eclipse/updates/3.7>
at
org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(Targ
etDefinitionResolver.java:190)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolver.resolveContent(Targ
etDefinitionResolver.java:109)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolverService.resolveFromA
rguments(TargetDefinitionResolverService.java:64)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolverService.getTargetDef
initionContent(TargetDefinitionResolverService.java:54)
at
org.eclipse.tycho.p2.target.TargetPlatformBuilderImpl.addTargetDefinitio
n(TargetPlatformBuilderImpl.java:289)
at
org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileCont
entToTargetPlatform(P2TargetPlatformResolver.java:331)
... 24 more
Caused by: org.eclipse.equinox.p2.core.ProvisionException: HTTP Proxy
Authentication Required:
http://download.eclipse.org/eclipse/updates/3.7/content.xml
<http://download.eclipse.org/eclipse/updates/3.7/content.xml>
at
org.eclipse.equinox.internal.p2.repository.CacheManager.createCache(Cach
eManager.java:192)
at
org.eclipse.tycho.p2.remote.TychoP2RepositoryCacheManager.createCache(Ty
choP2RepositoryCacheManager.java:62)
at
org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataReposi
toryFactory.getLocalFile(SimpleMetadataRepositoryFactory.java:66)
at
org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataReposi
toryFactory.load(SimpleMetadataRepositoryFactory.java:88)
at
org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryMa
nager.factoryLoad(MetadataRepositoryManager.java:57)
at
org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryMan
ager.loadRepository(AbstractRepositoryManager.java:758)
at
org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryMan
ager.loadRepository(AbstractRepositoryManager.java:651)
at
org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryMa
nager.loadRepository(MetadataRepositoryManager.java:96)
at
org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryMa
nager.loadRepository(MetadataRepositoryManager.java:92)
at
org.eclipse.tycho.p2.remote.RemoteMetadataRepositoryManager.loadReposito
ry(RemoteMetadataRepositoryManager.java:82)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(Targ
etDefinitionResolver.java:188)
... 29 more
Caused by: org.eclipse.ecf.filetransfer.BrowseFileTransferException:
Proxy auth required
at
org.eclipse.ecf.provider.filetransfer.browse.URLFileSystemBrowser.runReq
uest(URLFileSystemBrowser.java:129)
at
org.eclipse.ecf.provider.filetransfer.browse.AbstractFileSystemBrowser$D
irectoryJob.run(AbstractFileSystemBrowser.java:69)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)
channel stopped



Output from the console (OK):



Downloaded:
http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plug
in/1.3/aspectj-maven-plugin-1.3.jar
<http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plu
gin/1.3/aspectj-maven-plugin-1.3.jar> (37 KB at 63.6 KB/sec)

[INFO] Scanning for projects...

[INFO] Computing target platform for MavenProject:
<MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml

18-Feb-2013 14:58:28
org.apache.commons.httpclient.auth.AuthChallengeProcessor
selectAuthScheme

INFO: ntlm authentication scheme selected

18-Feb-2013 14:58:28 org.apache.commons.httpclient.HttpMethodDirector
processProxyAuthChallenge

INFO: No credentials available for NTLM <any realm>@iaccess.bp.com:80

[WARNING] De-selecting bundles in a target definition file is not
supported. See
http://wiki.eclipse.org/Tycho_Messages_Explained#Target_File_Include_Bun
dles for alternatives.

[INFO] Resolving dependencies of MavenProject:
<MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml

[WARNING] The following locally built units have been used to resolve
project dependencies:

<list of dependencies to local modules>

[INFO] Resolving class path of MavenProject:
<MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml

[WARNING] Dependency from C:\git\ok\<MODULE_ID_HIDDEN> to nested
classpath entry C:\git\ok\<OTHER_MODULE>\icons can not be represented in
Maven model and will not be visible to non-OSGi aware Maven plugins



Thank,

Vincent

Vincent Girard-Reydet - BP Gas Marketing Limited, 20 Canada Square,
London, E14 5NJ

BP Gas Marketing Limited, a company registered in England and Wales with
the company number 908982, VAT number GB 365 6789 95 and whose
registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16
7BP

This message may contain information that is privileged or confidential.
If you are not the intended recipient please delete it and inform the
sender immediately. Within the bounds of law, the UK based entities in
the Trading arm of the BP Group retain all emails and IMs and monitor
them to ensure compliance with their internal policies and for other
legitimate business purposes.
Sievers, Jan
2013-02-18 16:27:40 UTC
Permalink
are you using maven-encrypted passwords?
https://bugs.eclipse.org/bugs/show_bug.cgi?id=394023

are you sure you added proxy credentials in settings.xml [1] ?

other than that you may have hit an NTLM limitation in the outdated apache httpclient 3.x still used by p2.
here is the bug to upgrade to httpclient v4 which seems to indicate it has better support for NTLM:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449#c2

in this case I suggest to ask on p2-***@eclipse.org for any hints.

Regards
Jan

[1] http://maven.apache.org/settings.html#Proxies



From: tycho-user-***@eclipse.org [mailto:tycho-user-***@eclipse.org] On Behalf Of Girard-reydet, Vincent
Sent: Montag, 18. Februar 2013 16:07
To: tycho-***@eclipse.org
Subject: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy

Hi folks,

I'm struggling to configure a Maven build using Tycho on Jenkins behind my company's proxy. I have a set of modules that build perfectly well from the console (I just rebuilt everything using a fresh m2 repo and the build is completely successful). Now I' trying to automate it with Jenkins (see the full env spec at the end of the email). I have configured a Maven build in Jenkins. For the local and global settings, I'm pointing at the exact same settings file I'm using in the console. This settings file contains the proxy definition, with the username, password and everything (and working fine in console). The build is using a private Maven repo.
For my build, I'm using a target platform file (NOT using p2 repositories in the pom.xml), as per the below. This platform is resolved OK when building in console.

The Jenkins build uses correctly the proxy information to download all Maven plugins (including Tycho plugins). However, when Tycho starts resolving the platform, it somehow fails to collect the proxy information (which it should since TYCHO-279 is now closed). I tried all the workarounds described to circumvene TYCHO-279 (using the ECF proxy properties), I tried also using a Nexus repo, nothing worked. I'm a bit screwed new, it seems to be somehow Jenkins-specific but I don't' have a clue what's going on.

Would someone be able to help me investigate the problem?

Environment spec:
- Windows 7
- NTLM2 proxy
- Maven 3.0.4
- JDK 1.6.39 x86_64
- Jenkins v1.501 (out of the box install, running as a Windows service)
- Tycho v0.16

Platform configuration in parent pom:

      <plugin>
        <groupId>org.eclipse.tycho</groupId>
        <artifactId>target-platform-configuration</artifactId>
        <version>${tycho.version}</version>
        <configuration>
         <target>
          <artifact>
             <groupId>FAKE_GROUP_ID</groupId>
             <artifactId>target</artifactId>
             <version>1.0.0</version>
             <classifier>indigo</classifier>
          </artifact>
         </target>
          <environments>
            <environment>
              <os>win32</os>
              <ws>win32</ws>
              <arch>x86</arch>
            </environment>
            <environment>
              <os>win32</os>
              <ws>win32</ws>
              <arch>x86_64</arch>
            </environment>
          </environments>
        </configuration>
      </plugin>

Platform definition :
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?pde version="3.6"?>

<target name="indigo" sequenceNumber="14">
<locations>
<location includeAllPlatforms="false" includeMode="slicer" includeSource="false" type="InstallableUnit">
<unit id="org.eclipse.equinox.server.servletbridge.feature.group" version="1.0.100.v20110502-42DF9oB5865K5D"/>
<repository location="http://download.eclipse.org/eclipse/updates/3.7"/>
</location>
<location includeAllPlatforms="false" includeMode="slicer" includeSource="false" type="InstallableUnit">
<unit id="org.eclipse.ajdt.feature.group" version="2.2.0.e37x-RELEASE-20120704-0900"/>
<unit id="org.eclipse.equinox.weaving.sdk.feature.group" version="1.0.200.I20120427-0800"/>
<repository location="http://download.eclipse.org/tools/ajdt/37/update"/>
</location>
<location includeAllPlatforms="false" includeMode="slicer" includeSource="false" type="InstallableUnit">
<unit id="org.eclipse.sdk.ide" version="3.7.2.M20120208-0800"/>
<unit id="org.eclipse.emf.ecore.feature.group" version="2.7.1.v20120127-1122"/>
<unit id="org.eclipse.emf.sdk.feature.group" version="2.7.2.v20120130-0943"/>
<unit id="org.eclipse.emf.compare.feature.group" version="1.2.2.v20120214-0915"/>
<unit id="org.eclipse.emf.query.feature.group" version="1.5.0.v20110502-1600-218Z7w311A142A1142"/>
<unit id="org.eclipse.draw2d.feature.group" version="3.7.2.v20110927-2020-4617w3122212803131"/>
<unit id="org.eclipse.xsd.sdk.feature.group" version="2.7.1.v20120130-0943"/>
<unit id="org.eclipse.objectteams.otequinox.feature.group" version="2.0.2.201202051448"/>
<unit id="org.eclipse.equinox.sdk.feature.group" version="3.7.1.R37x_v20110907-7M7W8i8eNV4WsRkue-4Vq4J6pCyW"/>
<unit id="javax.transaction" version="1.1.1.v201105210645"/>

<unit id="org.eclipse.equinox.executable.feature.group" version="3.5.0.v20110530-7P7NFUFFLWUl76mart"/> 
<unit id="org.eclipse.jdt.feature.group" version="3.7.1.r371_v20110810-0800-7z8gFcoFMLfTabvKsR5Qm9rBGEBK"/>
<unit id="org.eclipse.rcp.sdk.id" version="3.7.1.M20110909-1335"/>

<repository location="http://download.eclipse.org/releases/indigo"/>
</location>
</locations>
<environment>
<arch>x86</arch>
</environment>
</target>

Error log in Jenkins (no error when building from console):

Downloaded: http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plugin/1.3/aspectj-maven-plugin-1.3.jar (37 KB at 63.6 KB/sec)
[INFO] Computing target platform for MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\jenkins_workspace\test\<MODULE_ID_HIDDEN>\pom.xml
18-Feb-2013 12:30:08 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 12:30:08 org.apache.commons.httpclient.HttpMethodDirector processProxyAuthChallenge
INFO: No credentials available for NTLM <any realm>@iaccess.bp.com:80
mavenExecutionResult exceptions not empty
message : Internal error: java.lang.RuntimeException: Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
cause : Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
Stack trace :
org.apache.maven.InternalErrorException: Internal error: java.lang.RuntimeException: Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
         at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:168)
         at org.jvnet.hudson.maven3.launcher.Maven3Launcher.main(Maven3Launcher.java:79)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:597)
         at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329)
         at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239)
         at org.jvnet.hudson.maven3.agent.Maven3Main.launch(Maven3Main.java:158)
         at hudson.maven.Maven3Builder.call(Maven3Builder.java:100)
         at hudson.maven.Maven3Builder.call(Maven3Builder.java:66)
         at hudson.remoting.UserRequest.perform(UserRequest.java:118)
         at hudson.remoting.UserRequest.perform(UserRequest.java:48)
         at hudson.remoting.Request$2.run(Request.java:326)
         at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
         at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.RuntimeException: Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
         at org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileContentToTargetPlatform(P2TargetPlatformResolver.java:338)
         at org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.computeTargetPlatform(P2TargetPlatformResolver.java:207)
         at org.eclipse.tycho.core.resolver.DefaultTychoDependencyResolver.resolveProject(DefaultTychoDependencyResolver.java:94)
         at org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.afterProjectsRead(TychoMavenLifecycleParticipant.java:82)
         at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:274)
         at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
         ... 19 more
Caused by: org.eclipse.tycho.p2.target.facade.TargetDefinitionResolutionException: Failed to load metadata repository from location http://download.eclipse.org/eclipse/updates/3.7
         at org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(TargetDefinitionResolver.java:190)
         at org.eclipse.tycho.p2.target.TargetDefinitionResolver.resolveContent(TargetDefinitionResolver.java:109)
         at org.eclipse.tycho.p2.target.TargetDefinitionResolverService.resolveFromArguments(TargetDefinitionResolverService.java:64)
         at org.eclipse.tycho.p2.target.TargetDefinitionResolverService.getTargetDefinitionContent(TargetDefinitionResolverService.java:54)
         at org.eclipse.tycho.p2.target.TargetPlatformBuilderImpl.addTargetDefinition(TargetPlatformBuilderImpl.java:289)
         at org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileContentToTargetPlatform(P2TargetPlatformResolver.java:331)
         ... 24 more
Caused by: org.eclipse.equinox.p2.core.ProvisionException: HTTP Proxy Authentication Required: http://download.eclipse.org/eclipse/updates/3.7/content.xml
         at org.eclipse.equinox.internal.p2.repository.CacheManager.createCache(CacheManager.java:192)
         at org.eclipse.tycho.p2.remote.TychoP2RepositoryCacheManager.createCache(TychoP2RepositoryCacheManager.java:62)
         at org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataRepositoryFactory.getLocalFile(SimpleMetadataRepositoryFactory.java:66)
         at org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataRepositoryFactory.load(SimpleMetadataRepositoryFactory.java:88)
         at org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.factoryLoad(MetadataRepositoryManager.java:57)
         at org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryManager.loadRepository(AbstractRepositoryManager.java:758)
         at org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryManager.loadRepository(AbstractRepositoryManager.java:651)
         at org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.loadRepository(MetadataRepositoryManager.java:96)
         at org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.loadRepository(MetadataRepositoryManager.java:92)
         at org.eclipse.tycho.p2.remote.RemoteMetadataRepositoryManager.loadRepository(RemoteMetadataRepositoryManager.java:82)
         at org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(TargetDefinitionResolver.java:188)
         ... 29 more
Caused by: org.eclipse.ecf.filetransfer.BrowseFileTransferException: Proxy auth required
         at org.eclipse.ecf.provider.filetransfer.browse.URLFileSystemBrowser.runRequest(URLFileSystemBrowser.java:129)
         at org.eclipse.ecf.provider.filetransfer.browse.AbstractFileSystemBrowser$DirectoryJob.run(AbstractFileSystemBrowser.java:69)
         at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)
channel stopped

Output from the console (OK):

Downloaded: http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plugin/1.3/aspectj-maven-plugin-1.3.jar (37 KB at 63.6 KB/sec)
[INFO] Scanning for projects...
[INFO] Computing target platform for MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
18-Feb-2013 14:58:28 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 14:58:28 org.apache.commons.httpclient.HttpMethodDirector processProxyAuthChallenge
INFO: No credentials available for NTLM <any realm>@iaccess.bp.com:80
[WARNING] De-selecting bundles in a target definition file is not supported. See http://wiki.eclipse.org/Tycho_Messages_Explained#Target_File_Include_Bundles for alternatives.
[INFO] Resolving dependencies of MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
[WARNING] The following locally built units have been used to resolve project dependencies:
<list of dependencies to local modules>
[INFO] Resolving class path of MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
[WARNING] Dependency from C:\git\ok\<MODULE_ID_HIDDEN> to nested classpath entry C:\git\ok\<OTHER_MODULE>\icons can not be represented in Maven model and will not be visible to non-OSGi aware Maven plugins

Thank,
Vincent
Vincent Girard-Reydet - BP Gas Marketing Limited, 20 Canada Square, London, E14 5NJ
BP Gas Marketing Limited, a company registered in England and Wales with the company number 908982, VAT number GB 365 6789 95 and whose registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16 7BP
This message may contain information that is privileged or confidential. If you are not the intended recipient please delete it and inform the sender immediately. Within the bounds of law, the UK based entities in the Trading arm of the BP Group retain all emails and IMs and monitor them to ensure compliance with their internal policies and for other legitimate business purposes.
Girard-reydet, Vincent
2013-02-18 17:49:44 UTC
Permalink
Hi Jan,

Thank you for your answer. No I'm not using encrypted passwords for now. I have triple-checked the settings and it's using the proxy (I've tried removing the section completely and the Jenkins fails to download the Maven plugins). Here is the proxy section I'm using (passwords, logins and hosts modified):

<proxies>
<proxy>
<id>bp</id>
<active>true</active>
<protocol>http</protocol>
<username>MYUSERNAME</username>
<password>MYPASSWORD</password>
<host>my.proxy.host</host>
<port>808080</port>
<!--<nonProxyHosts>local.net|some.host.com</nonProxyHosts>-->
</proxy>
</proxies>

Note that I have dropped the wagon-http-lightweight-2.2.jar in the ${MAVEN_HOME}/lib/ext folder of my Maven installation as well as in the folder of Jenkins, just to be sure.

But what I really don't understand is why it works in command line and not in Jenkins. I don't see where the difference is, as I'm using the same version of Maven and the same config file. I don't have enough knowledge of the platform resolution process to understand: where does Maven stop, where does Tycho stop and where does pure P2 code start in the platform resolution process, and how does the chain of transmission of credentials work? I thought that TYCHO-279 had solved the problem, but from the fix I was not able to completely understand the process.

To me it seems that Tycho should configure the proxy/credentials resolution in p2 before invoking it to resolve the platform, so the bug would be in the Tycho/Jenkins interaction, but I may be wrong.

To try the httpclient4, I have to consume the newest version of org.eclipse.ecf.filetransfer.feature, that's right? What would be the recommended way of doing (or is it what I have to ask to p2-***@eclipse.org)?

Vincent

Vincent Girard-Reydet - LNG Optimisation - BP Gas Marketing Limited, 20 Canada Square, London, E14 5NJ
t: +44 20 7948 5122
BP Gas Marketing Limited, a company registered in England and Wales with the company number 908982, VAT number GB 365 6789 95 and whose registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16 7BP
This message may contain information that is privileged or confidential. If you are not the intended recipient please delete it and inform the sender immediately. Within the bounds of law, the UK based entities in the Trading arm of the BP Group retain all emails and IMs and monitor them to ensure compliance with their internal policies and for other legitimate business purposes.


-----Original Message-----
From: tycho-user-***@eclipse.org [mailto:tycho-user-***@eclipse.org] On Behalf Of Sievers, Jan
Sent: 18 February 2013 16:28
To: Tycho user list
Subject: Re: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy

are you using maven-encrypted passwords?
https://bugs.eclipse.org/bugs/show_bug.cgi?id=394023

are you sure you added proxy credentials in settings.xml [1] ?

other than that you may have hit an NTLM limitation in the outdated apache httpclient 3.x still used by p2.
here is the bug to upgrade to httpclient v4 which seems to indicate it has better support for NTLM:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449#c2

in this case I suggest to ask on p2-***@eclipse.org for any hints.

Regards
Jan

[1] http://maven.apache.org/settings.html#Proxies



From: tycho-user-***@eclipse.org [mailto:tycho-user-***@eclipse.org] On Behalf Of Girard-reydet, Vincent
Sent: Montag, 18. Februar 2013 16:07
To: tycho-***@eclipse.org
Subject: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy

Hi folks,

I'm struggling to configure a Maven build using Tycho on Jenkins behind my company's proxy. I have a set of modules that build perfectly well from the console (I just rebuilt everything using a fresh m2 repo and the build is completely successful). Now I' trying to automate it with Jenkins (see the full env spec at the end of the email). I have configured a Maven build in Jenkins. For the local and global settings, I'm pointing at the exact same settings file I'm using in the console. This settings file contains the proxy definition, with the username, password and everything (and working fine in console). The build is using a private Maven repo.
For my build, I'm using a target platform file (NOT using p2 repositories in the pom.xml), as per the below. This platform is resolved OK when building in console.

The Jenkins build uses correctly the proxy information to download all Maven plugins (including Tycho plugins). However, when Tycho starts resolving the platform, it somehow fails to collect the proxy information (which it should since TYCHO-279 is now closed). I tried all the workarounds described to circumvene TYCHO-279 (using the ECF proxy properties), I tried also using a Nexus repo, nothing worked. I'm a bit screwed new, it seems to be somehow Jenkins-specific but I don't' have a clue what's going on.


Would someone be able to help me investigate the problem?

Environment spec:
- Windows 7
- NTLM2 proxy
- Maven 3.0.4
- JDK 1.6.39 x86_64
- Jenkins v1.501 (out of the box install, running as a Windows service)
- Tycho v0.16

Platform configuration in parent pom:

      <plugin>
        <groupId>org.eclipse.tycho</groupId>
        <artifactId>target-platform-configuration</artifactId>
        <version>${tycho.version}</version>
        <configuration>
         <target>
          <artifact>
             <groupId>FAKE_GROUP_ID</groupId>
             <artifactId>target</artifactId>
             <version>1.0.0</version>
             <classifier>indigo</classifier>
          </artifact>
         </target>
          <environments>
            <environment>
              <os>win32</os>
              <ws>win32</ws>
              <arch>x86</arch>
            </environment>
            <environment>
              <os>win32</os>
              <ws>win32</ws>
              <arch>x86_64</arch>
            </environment>
          </environments>
        </configuration>
      </plugin>

Platform definition :
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?pde version="3.6"?>

<target name="indigo" sequenceNumber="14"> <locations> <location includeAllPlatforms="false" includeMode="slicer" includeSource="false" type="InstallableUnit"> <unit id="org.eclipse.equinox.server.servletbridge.feature.group" version="1.0.100.v20110502-42DF9oB5865K5D"/>
<repository location="http://download.eclipse.org/eclipse/updates/3.7"/>
</location>
<location includeAllPlatforms="false" includeMode="slicer" includeSource="false" type="InstallableUnit"> <unit id="org.eclipse.ajdt.feature.group" version="2.2.0.e37x-RELEASE-20120704-0900"/>
<unit id="org.eclipse.equinox.weaving.sdk.feature.group" version="1.0.200.I20120427-0800"/>
<repository location="http://download.eclipse.org/tools/ajdt/37/update"/>
</location>
<location includeAllPlatforms="false" includeMode="slicer" includeSource="false" type="InstallableUnit"> <unit id="org.eclipse.sdk.ide" version="3.7.2.M20120208-0800"/> <unit id="org.eclipse.emf.ecore.feature.group" version="2.7.1.v20120127-1122"/> <unit id="org.eclipse.emf.sdk.feature.group" version="2.7.2.v20120130-0943"/> <unit id="org.eclipse.emf.compare.feature.group" version="1.2.2.v20120214-0915"/> <unit id="org.eclipse.emf.query.feature.group" version="1.5.0.v20110502-1600-218Z7w311A142A1142"/>
<unit id="org.eclipse.draw2d.feature.group" version="3.7.2.v20110927-2020-4617w3122212803131"/>
<unit id="org.eclipse.xsd.sdk.feature.group" version="2.7.1.v20120130-0943"/> <unit id="org.eclipse.objectteams.otequinox.feature.group" version="2.0.2.201202051448"/> <unit id="org.eclipse.equinox.sdk.feature.group" version="3.7.1.R37x_v20110907-7M7W8i8eNV4WsRkue-4Vq4J6pCyW"/>
<unit id="javax.transaction" version="1.1.1.v201105210645"/>

<unit id="org.eclipse.equinox.executable.feature.group" version="3.5.0.v20110530-7P7NFUFFLWUl76mart"/>
<unit id="org.eclipse.jdt.feature.group" version="3.7.1.r371_v20110810-0800-7z8gFcoFMLfTabvKsR5Qm9rBGEBK"/>
<unit id="org.eclipse.rcp.sdk.id" version="3.7.1.M20110909-1335"/>

<repository location="http://download.eclipse.org/releases/indigo"/>
</location>
</locations>
<environment>
<arch>x86</arch>
</environment>
</target>

Error log in Jenkins (no error when building from console):

Downloaded: http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plugin/1.3/aspectj-maven-plugin-1.3.jar (37 KB at 63.6 KB/sec) [INFO] Computing target platform for MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\jenkins_workspace\test\<MODULE_ID_HIDDEN>\pom.xml
18-Feb-2013 12:30:08 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 12:30:08 org.apache.commons.httpclient.HttpMethodDirector processProxyAuthChallenge
INFO: No credentials available for NTLM <any realm>@iaccess.bp.com:80 mavenExecutionResult exceptions not empty message : Internal error: java.lang.RuntimeException: Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target

cause : Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
Stack trace :
org.apache.maven.InternalErrorException: Internal error: java.lang.RuntimeException: Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
         at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:168)
         at org.jvnet.hudson.maven3.launcher.Maven3Launcher.main(Maven3Launcher.java:79)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:597)
         at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329)
         at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239)
         at org.jvnet.hudson.maven3.agent.Maven3Main.launch(Maven3Main.java:158)
         at hudson.maven.Maven3Builder.call(Maven3Builder.java:100)
         at hudson.maven.Maven3Builder.call(Maven3Builder.java:66)
         at hudson.remoting.UserRequest.perform(UserRequest.java:118)
         at hudson.remoting.UserRequest.perform(UserRequest.java:48)
         at hudson.remoting.Request$2.run(Request.java:326)
         at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
         at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.RuntimeException: Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
         at org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileContentToTargetPlatform(P2TargetPlatformResolver.java:338)
         at org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.computeTargetPlatform(P2TargetPlatformResolver.java:207)
         at org.eclipse.tycho.core.resolver.DefaultTychoDependencyResolver.resolveProject(DefaultTychoDependencyResolver.java:94)
         at org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.afterProjectsRead(TychoMavenLifecycleParticipant.java:82)
         at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:274)
         at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
         ... 19 more
Caused by: org.eclipse.tycho.p2.target.facade.TargetDefinitionResolutionException: Failed to load metadata repository from location http://download.eclipse.org/eclipse/updates/3.7
         at org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(TargetDefinitionResolver.java:190)
         at org.eclipse.tycho.p2.target.TargetDefinitionResolver.resolveContent(TargetDefinitionResolver.java:109)
         at org.eclipse.tycho.p2.target.TargetDefinitionResolverService.resolveFromArguments(TargetDefinitionResolverService.java:64)
         at org.eclipse.tycho.p2.target.TargetDefinitionResolverService.getTargetDefinitionContent(TargetDefinitionResolverService.java:54)
         at org.eclipse.tycho.p2.target.TargetPlatformBuilderImpl.addTargetDefinition(TargetPlatformBuilderImpl.java:289)
         at org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileContentToTargetPlatform(P2TargetPlatformResolver.java:331)
         ... 24 more
Caused by: org.eclipse.equinox.p2.core.ProvisionException: HTTP Proxy Authentication Required: http://download.eclipse.org/eclipse/updates/3.7/content.xml
         at org.eclipse.equinox.internal.p2.repository.CacheManager.createCache(CacheManager.java:192)
         at org.eclipse.tycho.p2.remote.TychoP2RepositoryCacheManager.createCache(TychoP2RepositoryCacheManager.java:62)
         at org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataRepositoryFactory.getLocalFile(SimpleMetadataRepositoryFactory.java:66)
         at org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataRepositoryFactory.load(SimpleMetadataRepositoryFactory.java:88)
         at org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.factoryLoad(MetadataRepositoryManager.java:57)
         at org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryManager.loadRepository(AbstractRepositoryManager.java:758)
         at org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryManager.loadRepository(AbstractRepositoryManager.java:651)
         at org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.loadRepository(MetadataRepositoryManager.java:96)
         at org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.loadRepository(MetadataRepositoryManager.java:92)
         at org.eclipse.tycho.p2.remote.RemoteMetadataRepositoryManager.loadRepository(RemoteMetadataRepositoryManager.java:82)
         at org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(TargetDefinitionResolver.java:188)
         ... 29 more
Caused by: org.eclipse.ecf.filetransfer.BrowseFileTransferException: Proxy auth required
         at org.eclipse.ecf.provider.filetransfer.browse.URLFileSystemBrowser.runRequest(URLFileSystemBrowser.java:129)
         at org.eclipse.ecf.provider.filetransfer.browse.AbstractFileSystemBrowser$DirectoryJob.run(AbstractFileSystemBrowser.java:69)
         at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)
channel stopped

Output from the console (OK):

Downloaded: http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plugin/1.3/aspectj-maven-plugin-1.3.jar (37 KB at 63.6 KB/sec) [INFO] Scanning for projects...
[INFO] Computing target platform for MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
18-Feb-2013 14:58:28 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 14:58:28 org.apache.commons.httpclient.HttpMethodDirector processProxyAuthChallenge
INFO: No credentials available for NTLM <any realm>@iaccess.bp.com:80 [WARNING] De-selecting bundles in a target definition file is not supported. See http://wiki.eclipse.org/Tycho_Messages_Explained#Target_File_Include_Bundles for alternatives.
[INFO] Resolving dependencies of MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
[WARNING] The following locally built units have been used to resolve project dependencies:
<list of dependencies to local modules>
[INFO] Resolving class path of MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
[WARNING] Dependency from C:\git\ok\<MODULE_ID_HIDDEN> to nested classpath entry C:\git\ok\<OTHER_MODULE>\icons can not be represented in Maven model and will not be visible to non-OSGi aware Maven plugins

Thank,
Vincent
Vincent Girard-Reydet - BP Gas Marketing Limited, 20 Canada Square, London, E14 5NJ BP Gas Marketing Limited, a company registered in England and Wales with the company number 908982, VAT number GB 365 6789 95 and whose registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16 7BP This message may contain information that is privileged or confidential. If you are not the intended recipient please delete it and inform the sender immediately. Within the bounds of law, the UK based entities in the Trading arm of the BP Group retain all emails and IMs and monitor them to ensure compliance with their internal policies and for other legitimate business purposes.
Sievers, Jan
2013-02-18 20:56:39 UTC
Permalink
- cross-posting to p2-dev -

tycho takes the maven HTTP proxy configuration from settings.xml and configures p2 accordingly (via proxyservice API in org.eclipse.core.net)
if resolution and download of maven plugins works but p2 resolution does not, this is a hint that either

a) tycho fails to transfer the proxy info to p2 or
b) p2 / org.eclipse.core.net does not support NTLM proxy authentication

since nobody reported a) yet I doubt this and suspect b)

@p2-devs any hints as to b) above?
https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449#c2 may be related.

@Vincent just in case using apache httpclient v4 would help, this is not up to you; it's a change which would have to be done in p2 (or at least in tycho)
Another thing you should try is to enable advanced HTTP tracing as per [1] and post the traces

Regards,
Jan

[1] http://wiki.eclipse.org/Equinox_p2_Reporting_Problems


-----Original Message-----
From: tycho-user-***@eclipse.org [mailto:tycho-user-***@eclipse.org] On Behalf Of Girard-reydet, Vincent
Sent: Montag, 18. Februar 2013 18:50
To: Tycho user list
Subject: Re: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy

Hi Jan,

Thank you for your answer. No I'm not using encrypted passwords for now. I have triple-checked the settings and it's using the proxy (I've tried removing the section completely and the Jenkins fails to download the Maven plugins). Here is the proxy section I'm using (passwords, logins and hosts modified):

<proxies>
<proxy>
<id>bp</id>
<active>true</active>
<protocol>http</protocol>
<username>MYUSERNAME</username>
<password>MYPASSWORD</password>
<host>my.proxy.host</host>
<port>808080</port>
<!--<nonProxyHosts>local.net|some.host.com</nonProxyHosts>-->
</proxy>
</proxies>

Note that I have dropped the wagon-http-lightweight-2.2.jar in the ${MAVEN_HOME}/lib/ext folder of my Maven installation as well as in the folder of Jenkins, just to be sure.

But what I really don't understand is why it works in command line and not in Jenkins. I don't see where the difference is, as I'm using the same version of Maven and the same config file. I don't have enough knowledge of the platform resolution process to understand: where does Maven stop, where does Tycho stop and where does pure P2 code start in the platform resolution process, and how does the chain of transmission of credentials work? I thought that TYCHO-279 had solved the problem, but from the fix I was not able to completely understand the process.

To me it seems that Tycho should configure the proxy/credentials resolution in p2 before invoking it to resolve the platform, so the bug would be in the Tycho/Jenkins interaction, but I may be wrong.

To try the httpclient4, I have to consume the newest version of org.eclipse.ecf.filetransfer.feature, that's right? What would be the recommended way of doing (or is it what I have to ask to p2-***@eclipse.org)?

Vincent

Vincent Girard-Reydet - LNG Optimisation - BP Gas Marketing Limited, 20 Canada Square, London, E14 5NJ
t: +44 20 7948 5122
BP Gas Marketing Limited, a company registered in England and Wales with the company number 908982, VAT number GB 365 6789 95 and whose registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16 7BP
This message may contain information that is privileged or confidential. If you are not the intended recipient please delete it and inform the sender immediately. Within the bounds of law, the UK based entities in the Trading arm of the BP Group retain all emails and IMs and monitor them to ensure compliance with their internal policies and for other legitimate business purposes.


-----Original Message-----
From: tycho-user-***@eclipse.org [mailto:tycho-user-***@eclipse.org] On Behalf Of Sievers, Jan
Sent: 18 February 2013 16:28
To: Tycho user list
Subject: Re: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy

are you using maven-encrypted passwords?
https://bugs.eclipse.org/bugs/show_bug.cgi?id=394023

are you sure you added proxy credentials in settings.xml [1] ?

other than that you may have hit an NTLM limitation in the outdated apache httpclient 3.x still used by p2.
here is the bug to upgrade to httpclient v4 which seems to indicate it has better support for NTLM:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449#c2

in this case I suggest to ask on p2-***@eclipse.org for any hints.

Regards
Jan

[1] http://maven.apache.org/settings.html#Proxies



From: tycho-user-***@eclipse.org [mailto:tycho-user-***@eclipse.org] On Behalf Of Girard-reydet, Vincent
Sent: Montag, 18. Februar 2013 16:07
To: tycho-***@eclipse.org
Subject: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy

Hi folks,

I'm struggling to configure a Maven build using Tycho on Jenkins behind my company's proxy. I have a set of modules that build perfectly well from the console (I just rebuilt everything using a fresh m2 repo and the build is completely successful). Now I' trying to automate it with Jenkins (see the full env spec at the end of the email). I have configured a Maven build in Jenkins. For the local and global settings, I'm pointing at the exact same settings file I'm using in the console. This settings file contains the proxy definition, with the username, password and everything (and working fine in console). The build is using a private Maven repo.
For my build, I'm using a target platform file (NOT using p2 repositories in the pom.xml), as per the below. This platform is resolved OK when building in console.

The Jenkins build uses correctly the proxy information to download all Maven plugins (including Tycho plugins). However, when Tycho starts resolving the platform, it somehow fails to collect the proxy information (which it should since TYCHO-279 is now closed). I tried all the workarounds described to circumvene TYCHO-279 (using the ECF proxy properties), I tried also using a Nexus repo, nothing worked. I'm a bit screwed new, it seems to be somehow Jenkins-specific but I don't' have a clue what's going on.


Would someone be able to help me investigate the problem?

Environment spec:
- Windows 7
- NTLM2 proxy
- Maven 3.0.4
- JDK 1.6.39 x86_64
- Jenkins v1.501 (out of the box install, running as a Windows service)
- Tycho v0.16

Platform configuration in parent pom:

      <plugin>
        <groupId>org.eclipse.tycho</groupId>
        <artifactId>target-platform-configuration</artifactId>
        <version>${tycho.version}</version>
        <configuration>
         <target>
          <artifact>
             <groupId>FAKE_GROUP_ID</groupId>
             <artifactId>target</artifactId>
             <version>1.0.0</version>
             <classifier>indigo</classifier>
          </artifact>
         </target>
          <environments>
            <environment>
              <os>win32</os>
              <ws>win32</ws>
              <arch>x86</arch>
            </environment>
            <environment>
              <os>win32</os>
              <ws>win32</ws>
              <arch>x86_64</arch>
            </environment>
          </environments>
        </configuration>
      </plugin>

Platform definition :
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?pde version="3.6"?>

<target name="indigo" sequenceNumber="14"> <locations> <location includeAllPlatforms="false" includeMode="slicer" includeSource="false" type="InstallableUnit"> <unit id="org.eclipse.equinox.server.servletbridge.feature.group" version="1.0.100.v20110502-42DF9oB5865K5D"/>
<repository location="http://download.eclipse.org/eclipse/updates/3.7"/>
</location>
<location includeAllPlatforms="false" includeMode="slicer" includeSource="false" type="InstallableUnit"> <unit id="org.eclipse.ajdt.feature.group" version="2.2.0.e37x-RELEASE-20120704-0900"/>
<unit id="org.eclipse.equinox.weaving.sdk.feature.group" version="1.0.200.I20120427-0800"/>
<repository location="http://download.eclipse.org/tools/ajdt/37/update"/>
</location>
<location includeAllPlatforms="false" includeMode="slicer" includeSource="false" type="InstallableUnit"> <unit id="org.eclipse.sdk.ide" version="3.7.2.M20120208-0800"/> <unit id="org.eclipse.emf.ecore.feature.group" version="2.7.1.v20120127-1122"/> <unit id="org.eclipse.emf.sdk.feature.group" version="2.7.2.v20120130-0943"/> <unit id="org.eclipse.emf.compare.feature.group" version="1.2.2.v20120214-0915"/> <unit id="org.eclipse.emf.query.feature.group" version="1.5.0.v20110502-1600-218Z7w311A142A1142"/>
<unit id="org.eclipse.draw2d.feature.group" version="3.7.2.v20110927-2020-4617w3122212803131"/>
<unit id="org.eclipse.xsd.sdk.feature.group" version="2.7.1.v20120130-0943"/> <unit id="org.eclipse.objectteams.otequinox.feature.group" version="2.0.2.201202051448"/> <unit id="org.eclipse.equinox.sdk.feature.group" version="3.7.1.R37x_v20110907-7M7W8i8eNV4WsRkue-4Vq4J6pCyW"/>
<unit id="javax.transaction" version="1.1.1.v201105210645"/>

<unit id="org.eclipse.equinox.executable.feature.group" version="3.5.0.v20110530-7P7NFUFFLWUl76mart"/>
<unit id="org.eclipse.jdt.feature.group" version="3.7.1.r371_v20110810-0800-7z8gFcoFMLfTabvKsR5Qm9rBGEBK"/>
<unit id="org.eclipse.rcp.sdk.id" version="3.7.1.M20110909-1335"/>

<repository location="http://download.eclipse.org/releases/indigo"/>
</location>
</locations>
<environment>
<arch>x86</arch>
</environment>
</target>

Error log in Jenkins (no error when building from console):

Downloaded: http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plugin/1.3/aspectj-maven-plugin-1.3.jar (37 KB at 63.6 KB/sec) [INFO] Computing target platform for MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\jenkins_workspace\test\<MODULE_ID_HIDDEN>\pom.xml
18-Feb-2013 12:30:08 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 12:30:08 org.apache.commons.httpclient.HttpMethodDirector processProxyAuthChallenge
INFO: No credentials available for NTLM <any realm>@iaccess.bp.com:80 mavenExecutionResult exceptions not empty message : Internal error: java.lang.RuntimeException: Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target

cause : Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
Stack trace :
org.apache.maven.InternalErrorException: Internal error: java.lang.RuntimeException: Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
         at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:168)
         at org.jvnet.hudson.maven3.launcher.Maven3Launcher.main(Maven3Launcher.java:79)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:597)
         at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329)
         at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239)
         at org.jvnet.hudson.maven3.agent.Maven3Main.launch(Maven3Main.java:158)
         at hudson.maven.Maven3Builder.call(Maven3Builder.java:100)
         at hudson.maven.Maven3Builder.call(Maven3Builder.java:66)
         at hudson.remoting.UserRequest.perform(UserRequest.java:118)
         at hudson.remoting.UserRequest.perform(UserRequest.java:48)
         at hudson.remoting.Request$2.run(Request.java:326)
         at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
         at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.RuntimeException: Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
         at org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileContentToTargetPlatform(P2TargetPlatformResolver.java:338)
         at org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.computeTargetPlatform(P2TargetPlatformResolver.java:207)
         at org.eclipse.tycho.core.resolver.DefaultTychoDependencyResolver.resolveProject(DefaultTychoDependencyResolver.java:94)
         at org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.afterProjectsRead(TychoMavenLifecycleParticipant.java:82)
         at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:274)
         at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
         ... 19 more
Caused by: org.eclipse.tycho.p2.target.facade.TargetDefinitionResolutionException: Failed to load metadata repository from location http://download.eclipse.org/eclipse/updates/3.7
         at org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(TargetDefinitionResolver.java:190)
         at org.eclipse.tycho.p2.target.TargetDefinitionResolver.resolveContent(TargetDefinitionResolver.java:109)
         at org.eclipse.tycho.p2.target.TargetDefinitionResolverService.resolveFromArguments(TargetDefinitionResolverService.java:64)
         at org.eclipse.tycho.p2.target.TargetDefinitionResolverService.getTargetDefinitionContent(TargetDefinitionResolverService.java:54)
         at org.eclipse.tycho.p2.target.TargetPlatformBuilderImpl.addTargetDefinition(TargetPlatformBuilderImpl.java:289)
         at org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileContentToTargetPlatform(P2TargetPlatformResolver.java:331)
         ... 24 more
Caused by: org.eclipse.equinox.p2.core.ProvisionException: HTTP Proxy Authentication Required: http://download.eclipse.org/eclipse/updates/3.7/content.xml
         at org.eclipse.equinox.internal.p2.repository.CacheManager.createCache(CacheManager.java:192)
         at org.eclipse.tycho.p2.remote.TychoP2RepositoryCacheManager.createCache(TychoP2RepositoryCacheManager.java:62)
         at org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataRepositoryFactory.getLocalFile(SimpleMetadataRepositoryFactory.java:66)
         at org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataRepositoryFactory.load(SimpleMetadataRepositoryFactory.java:88)
         at org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.factoryLoad(MetadataRepositoryManager.java:57)
         at org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryManager.loadRepository(AbstractRepositoryManager.java:758)
         at org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryManager.loadRepository(AbstractRepositoryManager.java:651)
         at org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.loadRepository(MetadataRepositoryManager.java:96)
         at org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.loadRepository(MetadataRepositoryManager.java:92)
         at org.eclipse.tycho.p2.remote.RemoteMetadataRepositoryManager.loadRepository(RemoteMetadataRepositoryManager.java:82)
         at org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(TargetDefinitionResolver.java:188)
         ... 29 more
Caused by: org.eclipse.ecf.filetransfer.BrowseFileTransferException: Proxy auth required
         at org.eclipse.ecf.provider.filetransfer.browse.URLFileSystemBrowser.runRequest(URLFileSystemBrowser.java:129)
         at org.eclipse.ecf.provider.filetransfer.browse.AbstractFileSystemBrowser$DirectoryJob.run(AbstractFileSystemBrowser.java:69)
         at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)
channel stopped

Output from the console (OK):

Downloaded: http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plugin/1.3/aspectj-maven-plugin-1.3.jar (37 KB at 63.6 KB/sec) [INFO] Scanning for projects...
[INFO] Computing target platform for MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
18-Feb-2013 14:58:28 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 14:58:28 org.apache.commons.httpclient.HttpMethodDirector processProxyAuthChallenge
INFO: No credentials available for NTLM <any realm>@iaccess.bp.com:80 [WARNING] De-selecting bundles in a target definition file is not supported. See http://wiki.eclipse.org/Tycho_Messages_Explained#Target_File_Include_Bundles for alternatives.
[INFO] Resolving dependencies of MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
[WARNING] The following locally built units have been used to resolve project dependencies:
<list of dependencies to local modules>
[INFO] Resolving class path of MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
[WARNING] Dependency from C:\git\ok\<MODULE_ID_HIDDEN> to nested classpath entry C:\git\ok\<OTHER_MODULE>\icons can not be represented in Maven model and will not be visible to non-OSGi aware Maven plugins

Thank,
Vincent
Vincent Girard-Reydet - BP Gas Marketing Limited, 20 Canada Square, London, E14 5NJ BP Gas Marketing Limited, a company registered in England and Wales with the company number 908982, VAT number GB 365 6789 95 and whose registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16 7BP This message may contain information that is privileged or confidential. If you are not the intended recipient please delete it and inform the sender immediately. Within the bounds of law, the UK based entities in the Trading arm of the BP Group retain all emails and IMs and monitor them to ensure compliance with their internal policies and for other legitimate business purposes.
Sievers, Jan
2013-02-19 08:45:24 UTC
Permalink
thanks Scott I think this will help to dig a little deeper.
I don't know how tycho handles the download...so I can't say exactly
what's happening here.
We delegate this to p2/ECF so it should work as if you would install/update with your IDE.
All we do is configure the HTTP proxy via org.eclipse.core.net (and register the default java.net.Authenticator, same as org.eclipse.core.net.ui does IIRC).

two follow-up questions:

1. is there any log/trace so we can check the fallback to the JRE proxy provider is actually happening in this case?
2. if the JRE proxy fallback happens, any chance different JDKs (OS/version) may have differing/no support for NTLM proxy authentication?

@Vincent another thing you can check to help us pin down the issue:
using the same HTTP proxy configurations in your IDE (in Window>Prefs), can you successfully update/install any bundles e.g. from eclipse.org ?
just to make sure I got this right: you say the build works fine on one machine but doesn't on another?
what's the difference between those 2 machines (OS, JDK) ?

If the httpclient v4 support is available but not consumed yet, ultimately we may decide to bundle httpclient v4 with tycho if that solves the issue.

Regards,
Jan


-----Original Message-----
From: p2-dev-***@eclipse.org [mailto:p2-dev-***@eclipse.org] On Behalf Of Scott Lewis
Sent: Montag, 18. Februar 2013 23:50
To: p2-***@eclipse.org
Subject: Re: [p2-dev] Tycho 0.16 + Jenkins + NTLM2 proxy

FYI,
- cross-posting to p2-dev -
tycho takes the maven HTTP proxy configuration from settings.xml and configures p2 accordingly (via proxyservice API in org.eclipse.core.net)
if resolution and download of maven plugins works but p2 resolution does not, this is a hint that either
a) tycho fails to transfer the proxy info to p2 or
b) p2 / org.eclipse.core.net does not support NTLM proxy authentication
since nobody reported a) yet I doubt this and suspect b)
I don't know how tycho handles the download...so I can't say exactly
what's happening here.

WRT b...p2 (via ECF filetransfer) does indeed support NTLMv2. There's a
long history to this...see [1]...but the upshot is that ECF filetransfer
has a workaround to not use the apache httpclient 3.1 provider, in the
event of NTLMv2 (since apache httpclient < 4.0 did not support NTLMv2,
and that's what ECF used until recently).
@p2-devs any hints as to b) above?
https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449#c2 may be related.
@Vincent just in case using apache httpclient v4 would help, this is not up to you; it's a change which would have to be done in p2 (or at least in tycho)
No...it's a change that's underway...intended for Kepler: i.e. see this
enhancement [2]. The desire here is that apache httpclient 4.x
reportedly has support for ntlmv2 proxies, where httpclient 3.1 did
not. We have a completed provider based upon httpclient 4.x, and would
appreciate some help with testing against various proxy configurations
in coming integration builds.

But again...since I don't know how/what tycho does WRT downloading, this
might not address your proxy issue.

Thanks,

Scott

[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=252002
[2] https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449
Another thing you should try is to enable advanced HTTP tracing as per [1] and post the traces
Regards,
Jan
[1] http://wiki.eclipse.org/Equinox_p2_Reporting_Problems
-----Original Message-----
Sent: Montag, 18. Februar 2013 18:50
To: Tycho user list
Subject: Re: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy
Hi Jan,
<proxies>
<proxy>
<id>bp</id>
<active>true</active>
<protocol>http</protocol>
<username>MYUSERNAME</username>
<password>MYPASSWORD</password>
<host>my.proxy.host</host>
<port>808080</port>
<!--<nonProxyHosts>local.net|some.host.com</nonProxyHosts>-->
</proxy>
</proxies>
Note that I have dropped the wagon-http-lightweight-2.2.jar in the ${MAVEN_HOME}/lib/ext folder of my Maven installation as well as in the folder of Jenkins, just to be sure.
But what I really don't understand is why it works in command line and not in Jenkins. I don't see where the difference is, as I'm using the same version of Maven and the same config file. I don't have enough knowledge of the platform resolution process to understand: where does Maven stop, where does Tycho stop and where does pure P2 code start in the platform resolution process, and how does the chain of transmission of credentials work? I thought that TYCHO-279 had solved the problem, but from the fix I was not able to completely understand the process.
To me it seems that Tycho should configure the proxy/credentials resolution in p2 before invoking it to resolve the platform, so the bug would be in the Tycho/Jenkins interaction, but I may be wrong.
Vincent
Vincent Girard-Reydet - LNG Optimisation - BP Gas Marketing Limited, 20 Canada Square, London, E14 5NJ
t: +44 20 7948 5122
BP Gas Marketing Limited, a company registered in England and Wales with the company number 908982, VAT number GB 365 6789 95 and whose registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16 7BP
This message may contain information that is privileged or confidential. If you are not the intended recipient please delete it and inform the sender immediately. Within the bounds of law, the UK based entities in the Trading arm of the BP Group retain all emails and IMs and monitor them to ensure compliance with their internal policies and for other legitimate business purposes.
-----Original Message-----
Sent: 18 February 2013 16:28
To: Tycho user list
Subject: Re: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy
are you using maven-encrypted passwords?
https://bugs.eclipse.org/bugs/show_bug.cgi?id=394023
are you sure you added proxy credentials in settings.xml [1] ?
other than that you may have hit an NTLM limitation in the outdated apache httpclient 3.x still used by p2.
https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449#c2
Regards
Jan
[1] http://maven.apache.org/settings.html#Proxies
Sent: Montag, 18. Februar 2013 16:07
Subject: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy
Hi folks,
I'm struggling to configure a Maven build using Tycho on Jenkins behind my company's proxy. I have a set of modules that build perfectly well from the console (I just rebuilt everything using a fresh m2 repo and the build is completely successful). Now I' trying to automate it with Jenkins (see the full env spec at the end of the email). I have configured a Maven build in Jenkins. For the local and global settings, I'm pointing at the exact same settings file I'm using in the console. This settings file contains the proxy definition, with the username, password and everything (and working fine in console). The build is using a private Maven repo.
For my build, I'm using a target platform file (NOT using p2 repositories in the pom.xml), as per the below. This platform is resolved OK when building in console.
The Jenkins build uses correctly the proxy information to download all Maven plugins (including Tycho plugins). However, when Tycho starts resolving the platform, it somehow fails to collect the proxy information (which it should since TYCHO-279 is now closed). I tried all the workarounds described to circumvene TYCHO-279 (using the ECF proxy properties), I tried also using a Nexus repo, nothing worked. I'm a bit screwed new, it seems to be somehow Jenkins-specific but I don't' have a clue what's going on.
Would someone be able to help me investigate the problem?
- Windows 7
- NTLM2 proxy
- Maven 3.0.4
- JDK 1.6.39 x86_64
- Jenkins v1.501 (out of the box install, running as a Windows service)
- Tycho v0.16
<plugin>
<groupId>org.eclipse.tycho</groupId>
<artifactId>target-platform-configuration</artifactId>
<version>${tycho.version}</version>
<configuration>
<target>
<artifact>
<groupId>FAKE_GROUP_ID</groupId>
<artifactId>target</artifactId>
<version>1.0.0</version>
<classifier>indigo</classifier>
</artifact>
</target>
<environments>
<environment>
<os>win32</os>
<ws>win32</ws>
<arch>x86</arch>
</environment>
<environment>
<os>win32</os>
<ws>win32</ws>
<arch>x86_64</arch>
</environment>
</environments>
</configuration>
</plugin>
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?pde version="3.6"?>
<target name="indigo" sequenceNumber="14"> <locations> <location includeAllPlatforms="false" includeMode="slicer" includeSource="false" type="InstallableUnit"> <unit id="org.eclipse.equinox.server.servletbridge.feature.group" version="1.0.100.v20110502-42DF9oB5865K5D"/>
<repository location="http://download.eclipse.org/eclipse/updates/3.7"/>
</location>
<location includeAllPlatforms="false" includeMode="slicer" includeSource="false" type="InstallableUnit"> <unit id="org.eclipse.ajdt.feature.group" version="2.2.0.e37x-RELEASE-20120704-0900"/>
<unit id="org.eclipse.equinox.weaving.sdk.feature.group" version="1.0.200.I20120427-0800"/>
<repository location="http://download.eclipse.org/tools/ajdt/37/update"/>
</location>
<location includeAllPlatforms="false" includeMode="slicer" includeSource="false" type="InstallableUnit"> <unit id="org.eclipse.sdk.ide" version="3.7.2.M20120208-0800"/> <unit id="org.eclipse.emf.ecore.feature.group" version="2.7.1.v20120127-1122"/> <unit id="org.eclipse.emf.sdk.feature.group" version="2.7.2.v20120130-0943"/> <unit id="org.eclipse.emf.compare.feature.group" version="1.2.2.v20120214-0915"/> <unit id="org.eclipse.emf.query.feature.group" version="1.5.0.v20110502-1600-218Z7w311A142A1142"/>
<unit id="org.eclipse.draw2d.feature.group" version="3.7.2.v20110927-2020-4617w3122212803131"/>
<unit id="org.eclipse.xsd.sdk.feature.group" version="2.7.1.v20120130-0943"/> <unit id="org.eclipse.objectteams.otequinox.feature.group" version="2.0.2.201202051448"/> <unit id="org.eclipse.equinox.sdk.feature.group" version="3.7.1.R37x_v20110907-7M7W8i8eNV4WsRkue-4Vq4J6pCyW"/>
<unit id="javax.transaction" version="1.1.1.v201105210645"/>
<unit id="org.eclipse.equinox.executable.feature.group" version="3.5.0.v20110530-7P7NFUFFLWUl76mart"/>
<unit id="org.eclipse.jdt.feature.group" version="3.7.1.r371_v20110810-0800-7z8gFcoFMLfTabvKsR5Qm9rBGEBK"/>
<unit id="org.eclipse.rcp.sdk.id" version="3.7.1.M20110909-1335"/>
<repository location="http://download.eclipse.org/releases/indigo"/>
</location>
</locations>
<environment>
<arch>x86</arch>
</environment>
</target>
18-Feb-2013 12:30:08 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 12:30:08 org.apache.commons.httpclient.HttpMethodDirector processProxyAuthChallenge
cause : Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
org.apache.maven.InternalErrorException: Internal error: java.lang.RuntimeException: Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:168)
at org.jvnet.hudson.maven3.launcher.Maven3Launcher.main(Maven3Launcher.java:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239)
at org.jvnet.hudson.maven3.agent.Maven3Main.launch(Maven3Main.java:158)
at hudson.maven.Maven3Builder.call(Maven3Builder.java:100)
at hudson.maven.Maven3Builder.call(Maven3Builder.java:66)
at hudson.remoting.UserRequest.perform(UserRequest.java:118)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:326)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.RuntimeException: Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
at org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileContentToTargetPlatform(P2TargetPlatformResolver.java:338)
at org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.computeTargetPlatform(P2TargetPlatformResolver.java:207)
at org.eclipse.tycho.core.resolver.DefaultTychoDependencyResolver.resolveProject(DefaultTychoDependencyResolver.java:94)
at org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.afterProjectsRead(TychoMavenLifecycleParticipant.java:82)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:274)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
... 19 more
Caused by: org.eclipse.tycho.p2.target.facade.TargetDefinitionResolutionException: Failed to load metadata repository from location http://download.eclipse.org/eclipse/updates/3.7
at org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(TargetDefinitionResolver.java:190)
at org.eclipse.tycho.p2.target.TargetDefinitionResolver.resolveContent(TargetDefinitionResolver.java:109)
at org.eclipse.tycho.p2.target.TargetDefinitionResolverService.resolveFromArguments(TargetDefinitionResolverService.java:64)
at org.eclipse.tycho.p2.target.TargetDefinitionResolverService.getTargetDefinitionContent(TargetDefinitionResolverService.java:54)
at org.eclipse.tycho.p2.target.TargetPlatformBuilderImpl.addTargetDefinition(TargetPlatformBuilderImpl.java:289)
at org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileContentToTargetPlatform(P2TargetPlatformResolver.java:331)
... 24 more
Caused by: org.eclipse.equinox.p2.core.ProvisionException: HTTP Proxy Authentication Required: http://download.eclipse.org/eclipse/updates/3.7/content.xml
at org.eclipse.equinox.internal.p2.repository.CacheManager.createCache(CacheManager.java:192)
at org.eclipse.tycho.p2.remote.TychoP2RepositoryCacheManager.createCache(TychoP2RepositoryCacheManager.java:62)
at org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataRepositoryFactory.getLocalFile(SimpleMetadataRepositoryFactory.java:66)
at org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataRepositoryFactory.load(SimpleMetadataRepositoryFactory.java:88)
at org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.factoryLoad(MetadataRepositoryManager.java:57)
at org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryManager.loadRepository(AbstractRepositoryManager.java:758)
at org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryManager.loadRepository(AbstractRepositoryManager.java:651)
at org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.loadRepository(MetadataRepositoryManager.java:96)
at org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.loadRepository(MetadataRepositoryManager.java:92)
at org.eclipse.tycho.p2.remote.RemoteMetadataRepositoryManager.loadRepository(RemoteMetadataRepositoryManager.java:82)
at org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(TargetDefinitionResolver.java:188)
... 29 more
Caused by: org.eclipse.ecf.filetransfer.BrowseFileTransferException: Proxy auth required
at org.eclipse.ecf.provider.filetransfer.browse.URLFileSystemBrowser.runRequest(URLFileSystemBrowser.java:129)
at org.eclipse.ecf.provider.filetransfer.browse.AbstractFileSystemBrowser$DirectoryJob.run(AbstractFileSystemBrowser.java:69)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)
channel stopped
Downloaded: http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plugin/1.3/aspectj-maven-plugin-1.3.jar (37 KB at 63.6 KB/sec) [INFO] Scanning for projects...
18-Feb-2013 14:58:28 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 14:58:28 org.apache.commons.httpclient.HttpMethodDirector processProxyAuthChallenge
<list of dependencies to local modules>
[WARNING] Dependency from C:\git\ok\<MODULE_ID_HIDDEN> to nested classpath entry C:\git\ok\<OTHER_MODULE>\icons can not be represented in Maven model and will not be visible to non-OSGi aware Maven plugins
Thank,
Vincent
Vincent Girard-Reydet - BP Gas Marketing Limited, 20 Canada Square, London, E14 5NJ BP Gas Marketing Limited, a company registered in England and Wales with the company number 908982, VAT number GB 365 6789 95 and whose registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16 7BP This message may contain information that is privileged or confidential. If you are not the intended recipient please delete it and inform the sender immediately. Within the bounds of law, the UK based entities in the Trading arm of the BP Group retain all emails and IMs and monitor them to ensure compliance with their internal policies and for other legitimate business purposes.
_______________________________________________
tycho-user mailing list
https://dev.eclipse.org/mailman/listinfo/tycho-user
_______________________________________________
tycho-user mailing list
https://dev.eclipse.org/mailman/listinfo/tycho-user
_______________________________________________
p2-dev mailing list
https://dev.eclipse.org/mailman/listinfo/p2-dev
Sievers, Jan
2013-02-18 21:01:20 UTC
Permalink
also, run mvn with debug options (-X -e) and post the traces so we can check if the proxy information is passed on correctly to p2

take care to anonymize the logs before posting

Jan

-----Original Message-----
From: tycho-user-***@eclipse.org [mailto:tycho-user-***@eclipse.org] On Behalf Of Girard-reydet, Vincent
Sent: Montag, 18. Februar 2013 18:50
To: Tycho user list
Subject: Re: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy

Hi Jan,

Thank you for your answer. No I'm not using encrypted passwords for now. I have triple-checked the settings and it's using the proxy (I've tried removing the section completely and the Jenkins fails to download the Maven plugins). Here is the proxy section I'm using (passwords, logins and hosts modified):

<proxies>
<proxy>
<id>bp</id>
<active>true</active>
<protocol>http</protocol>
<username>MYUSERNAME</username>
<password>MYPASSWORD</password>
<host>my.proxy.host</host>
<port>808080</port>
<!--<nonProxyHosts>local.net|some.host.com</nonProxyHosts>-->
</proxy>
</proxies>

Note that I have dropped the wagon-http-lightweight-2.2.jar in the ${MAVEN_HOME}/lib/ext folder of my Maven installation as well as in the folder of Jenkins, just to be sure.

But what I really don't understand is why it works in command line and not in Jenkins. I don't see where the difference is, as I'm using the same version of Maven and the same config file. I don't have enough knowledge of the platform resolution process to understand: where does Maven stop, where does Tycho stop and where does pure P2 code start in the platform resolution process, and how does the chain of transmission of credentials work? I thought that TYCHO-279 had solved the problem, but from the fix I was not able to completely understand the process.

To me it seems that Tycho should configure the proxy/credentials resolution in p2 before invoking it to resolve the platform, so the bug would be in the Tycho/Jenkins interaction, but I may be wrong.

To try the httpclient4, I have to consume the newest version of org.eclipse.ecf.filetransfer.feature, that's right? What would be the recommended way of doing (or is it what I have to ask to p2-***@eclipse.org)?

Vincent

Vincent Girard-Reydet - LNG Optimisation - BP Gas Marketing Limited, 20 Canada Square, London, E14 5NJ
t: +44 20 7948 5122
BP Gas Marketing Limited, a company registered in England and Wales with the company number 908982, VAT number GB 365 6789 95 and whose registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16 7BP
This message may contain information that is privileged or confidential. If you are not the intended recipient please delete it and inform the sender immediately. Within the bounds of law, the UK based entities in the Trading arm of the BP Group retain all emails and IMs and monitor them to ensure compliance with their internal policies and for other legitimate business purposes.


-----Original Message-----
From: tycho-user-***@eclipse.org [mailto:tycho-user-***@eclipse.org] On Behalf Of Sievers, Jan
Sent: 18 February 2013 16:28
To: Tycho user list
Subject: Re: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy

are you using maven-encrypted passwords?
https://bugs.eclipse.org/bugs/show_bug.cgi?id=394023

are you sure you added proxy credentials in settings.xml [1] ?

other than that you may have hit an NTLM limitation in the outdated apache httpclient 3.x still used by p2.
here is the bug to upgrade to httpclient v4 which seems to indicate it has better support for NTLM:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449#c2

in this case I suggest to ask on p2-***@eclipse.org for any hints.

Regards
Jan

[1] http://maven.apache.org/settings.html#Proxies



From: tycho-user-***@eclipse.org [mailto:tycho-user-***@eclipse.org] On Behalf Of Girard-reydet, Vincent
Sent: Montag, 18. Februar 2013 16:07
To: tycho-***@eclipse.org
Subject: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy

Hi folks,

I'm struggling to configure a Maven build using Tycho on Jenkins behind my company's proxy. I have a set of modules that build perfectly well from the console (I just rebuilt everything using a fresh m2 repo and the build is completely successful). Now I' trying to automate it with Jenkins (see the full env spec at the end of the email). I have configured a Maven build in Jenkins. For the local and global settings, I'm pointing at the exact same settings file I'm using in the console. This settings file contains the proxy definition, with the username, password and everything (and working fine in console). The build is using a private Maven repo.
For my build, I'm using a target platform file (NOT using p2 repositories in the pom.xml), as per the below. This platform is resolved OK when building in console.

The Jenkins build uses correctly the proxy information to download all Maven plugins (including Tycho plugins). However, when Tycho starts resolving the platform, it somehow fails to collect the proxy information (which it should since TYCHO-279 is now closed). I tried all the workarounds described to circumvene TYCHO-279 (using the ECF proxy properties), I tried also using a Nexus repo, nothing worked. I'm a bit screwed new, it seems to be somehow Jenkins-specific but I don't' have a clue what's going on.


Would someone be able to help me investigate the problem?

Environment spec:
- Windows 7
- NTLM2 proxy
- Maven 3.0.4
- JDK 1.6.39 x86_64
- Jenkins v1.501 (out of the box install, running as a Windows service)
- Tycho v0.16

Platform configuration in parent pom:

      <plugin>
        <groupId>org.eclipse.tycho</groupId>
        <artifactId>target-platform-configuration</artifactId>
        <version>${tycho.version}</version>
        <configuration>
         <target>
          <artifact>
             <groupId>FAKE_GROUP_ID</groupId>
             <artifactId>target</artifactId>
             <version>1.0.0</version>
             <classifier>indigo</classifier>
          </artifact>
         </target>
          <environments>
            <environment>
              <os>win32</os>
              <ws>win32</ws>
              <arch>x86</arch>
            </environment>
            <environment>
              <os>win32</os>
              <ws>win32</ws>
              <arch>x86_64</arch>
            </environment>
          </environments>
        </configuration>
      </plugin>

Platform definition :
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?pde version="3.6"?>

<target name="indigo" sequenceNumber="14"> <locations> <location includeAllPlatforms="false" includeMode="slicer" includeSource="false" type="InstallableUnit"> <unit id="org.eclipse.equinox.server.servletbridge.feature.group" version="1.0.100.v20110502-42DF9oB5865K5D"/>
<repository location="http://download.eclipse.org/eclipse/updates/3.7"/>
</location>
<location includeAllPlatforms="false" includeMode="slicer" includeSource="false" type="InstallableUnit"> <unit id="org.eclipse.ajdt.feature.group" version="2.2.0.e37x-RELEASE-20120704-0900"/>
<unit id="org.eclipse.equinox.weaving.sdk.feature.group" version="1.0.200.I20120427-0800"/>
<repository location="http://download.eclipse.org/tools/ajdt/37/update"/>
</location>
<location includeAllPlatforms="false" includeMode="slicer" includeSource="false" type="InstallableUnit"> <unit id="org.eclipse.sdk.ide" version="3.7.2.M20120208-0800"/> <unit id="org.eclipse.emf.ecore.feature.group" version="2.7.1.v20120127-1122"/> <unit id="org.eclipse.emf.sdk.feature.group" version="2.7.2.v20120130-0943"/> <unit id="org.eclipse.emf.compare.feature.group" version="1.2.2.v20120214-0915"/> <unit id="org.eclipse.emf.query.feature.group" version="1.5.0.v20110502-1600-218Z7w311A142A1142"/>
<unit id="org.eclipse.draw2d.feature.group" version="3.7.2.v20110927-2020-4617w3122212803131"/>
<unit id="org.eclipse.xsd.sdk.feature.group" version="2.7.1.v20120130-0943"/> <unit id="org.eclipse.objectteams.otequinox.feature.group" version="2.0.2.201202051448"/> <unit id="org.eclipse.equinox.sdk.feature.group" version="3.7.1.R37x_v20110907-7M7W8i8eNV4WsRkue-4Vq4J6pCyW"/>
<unit id="javax.transaction" version="1.1.1.v201105210645"/>

<unit id="org.eclipse.equinox.executable.feature.group" version="3.5.0.v20110530-7P7NFUFFLWUl76mart"/>
<unit id="org.eclipse.jdt.feature.group" version="3.7.1.r371_v20110810-0800-7z8gFcoFMLfTabvKsR5Qm9rBGEBK"/>
<unit id="org.eclipse.rcp.sdk.id" version="3.7.1.M20110909-1335"/>

<repository location="http://download.eclipse.org/releases/indigo"/>
</location>
</locations>
<environment>
<arch>x86</arch>
</environment>
</target>

Error log in Jenkins (no error when building from console):

Downloaded: http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plugin/1.3/aspectj-maven-plugin-1.3.jar (37 KB at 63.6 KB/sec) [INFO] Computing target platform for MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\jenkins_workspace\test\<MODULE_ID_HIDDEN>\pom.xml
18-Feb-2013 12:30:08 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 12:30:08 org.apache.commons.httpclient.HttpMethodDirector processProxyAuthChallenge
INFO: No credentials available for NTLM <any realm>@iaccess.bp.com:80 mavenExecutionResult exceptions not empty message : Internal error: java.lang.RuntimeException: Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target

cause : Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
Stack trace :
org.apache.maven.InternalErrorException: Internal error: java.lang.RuntimeException: Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
         at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:168)
         at org.jvnet.hudson.maven3.launcher.Maven3Launcher.main(Maven3Launcher.java:79)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:597)
         at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329)
         at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239)
         at org.jvnet.hudson.maven3.agent.Maven3Main.launch(Maven3Main.java:158)
         at hudson.maven.Maven3Builder.call(Maven3Builder.java:100)
         at hudson.maven.Maven3Builder.call(Maven3Builder.java:66)
         at hudson.remoting.UserRequest.perform(UserRequest.java:118)
         at hudson.remoting.UserRequest.perform(UserRequest.java:48)
         at hudson.remoting.Request$2.run(Request.java:326)
         at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
         at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.RuntimeException: Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
         at org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileContentToTargetPlatform(P2TargetPlatformResolver.java:338)
         at org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.computeTargetPlatform(P2TargetPlatformResolver.java:207)
         at org.eclipse.tycho.core.resolver.DefaultTychoDependencyResolver.resolveProject(DefaultTychoDependencyResolver.java:94)
         at org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.afterProjectsRead(TychoMavenLifecycleParticipant.java:82)
         at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:274)
         at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
         ... 19 more
Caused by: org.eclipse.tycho.p2.target.facade.TargetDefinitionResolutionException: Failed to load metadata repository from location http://download.eclipse.org/eclipse/updates/3.7
         at org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(TargetDefinitionResolver.java:190)
         at org.eclipse.tycho.p2.target.TargetDefinitionResolver.resolveContent(TargetDefinitionResolver.java:109)
         at org.eclipse.tycho.p2.target.TargetDefinitionResolverService.resolveFromArguments(TargetDefinitionResolverService.java:64)
         at org.eclipse.tycho.p2.target.TargetDefinitionResolverService.getTargetDefinitionContent(TargetDefinitionResolverService.java:54)
         at org.eclipse.tycho.p2.target.TargetPlatformBuilderImpl.addTargetDefinition(TargetPlatformBuilderImpl.java:289)
         at org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileContentToTargetPlatform(P2TargetPlatformResolver.java:331)
         ... 24 more
Caused by: org.eclipse.equinox.p2.core.ProvisionException: HTTP Proxy Authentication Required: http://download.eclipse.org/eclipse/updates/3.7/content.xml
         at org.eclipse.equinox.internal.p2.repository.CacheManager.createCache(CacheManager.java:192)
         at org.eclipse.tycho.p2.remote.TychoP2RepositoryCacheManager.createCache(TychoP2RepositoryCacheManager.java:62)
         at org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataRepositoryFactory.getLocalFile(SimpleMetadataRepositoryFactory.java:66)
         at org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataRepositoryFactory.load(SimpleMetadataRepositoryFactory.java:88)
         at org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.factoryLoad(MetadataRepositoryManager.java:57)
         at org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryManager.loadRepository(AbstractRepositoryManager.java:758)
         at org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryManager.loadRepository(AbstractRepositoryManager.java:651)
         at org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.loadRepository(MetadataRepositoryManager.java:96)
         at org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.loadRepository(MetadataRepositoryManager.java:92)
         at org.eclipse.tycho.p2.remote.RemoteMetadataRepositoryManager.loadRepository(RemoteMetadataRepositoryManager.java:82)
         at org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(TargetDefinitionResolver.java:188)
         ... 29 more
Caused by: org.eclipse.ecf.filetransfer.BrowseFileTransferException: Proxy auth required
         at org.eclipse.ecf.provider.filetransfer.browse.URLFileSystemBrowser.runRequest(URLFileSystemBrowser.java:129)
         at org.eclipse.ecf.provider.filetransfer.browse.AbstractFileSystemBrowser$DirectoryJob.run(AbstractFileSystemBrowser.java:69)
         at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)
channel stopped

Output from the console (OK):

Downloaded: http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plugin/1.3/aspectj-maven-plugin-1.3.jar (37 KB at 63.6 KB/sec) [INFO] Scanning for projects...
[INFO] Computing target platform for MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
18-Feb-2013 14:58:28 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 14:58:28 org.apache.commons.httpclient.HttpMethodDirector processProxyAuthChallenge
INFO: No credentials available for NTLM <any realm>@iaccess.bp.com:80 [WARNING] De-selecting bundles in a target definition file is not supported. See http://wiki.eclipse.org/Tycho_Messages_Explained#Target_File_Include_Bundles for alternatives.
[INFO] Resolving dependencies of MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
[WARNING] The following locally built units have been used to resolve project dependencies:
<list of dependencies to local modules>
[INFO] Resolving class path of MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
[WARNING] Dependency from C:\git\ok\<MODULE_ID_HIDDEN> to nested classpath entry C:\git\ok\<OTHER_MODULE>\icons can not be represented in Maven model and will not be visible to non-OSGi aware Maven plugins

Thank,
Vincent
Vincent Girard-Reydet - BP Gas Marketing Limited, 20 Canada Square, London, E14 5NJ BP Gas Marketing Limited, a company registered in England and Wales with the company number 908982, VAT number GB 365 6789 95 and whose registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16 7BP This message may contain information that is privileged or confidential. If you are not the intended recipient please delete it and inform the sender immediately. Within the bounds of law, the UK based entities in the Trading arm of the BP Group retain all emails and IMs and monitor them to ensure compliance with their internal policies and for other legitimate business purposes.
Sievers, Jan
2013-02-19 09:26:16 UTC
Permalink
after I found and read [1] it seems to me you should specify

-Dorg.eclipse.ecf.provider.filetransfer.excludeContributors=org.eclipse.ecf.provider.filetransfer.httpclient

in order to disable the apache httpclient v3 support (which lacks NTLM proxy support) and fall back to the JRE HTTP provider instead.

Regards,
Jan

[1] http://wiki.eclipse.org/ECF_Filetransfer_Support_for_NTLMv2_Proxies



-----Original Message-----
From: tycho-user-***@eclipse.org [mailto:tycho-user-***@eclipse.org] On Behalf Of Girard-reydet, Vincent
Sent: Montag, 18. Februar 2013 18:50
To: Tycho user list
Subject: Re: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy

Hi Jan,

Thank you for your answer. No I'm not using encrypted passwords for now. I have triple-checked the settings and it's using the proxy (I've tried removing the section completely and the Jenkins fails to download the Maven plugins). Here is the proxy section I'm using (passwords, logins and hosts modified):

<proxies>
<proxy>
<id>bp</id>
<active>true</active>
<protocol>http</protocol>
<username>MYUSERNAME</username>
<password>MYPASSWORD</password>
<host>my.proxy.host</host>
<port>808080</port>
<!--<nonProxyHosts>local.net|some.host.com</nonProxyHosts>-->
</proxy>
</proxies>

Note that I have dropped the wagon-http-lightweight-2.2.jar in the ${MAVEN_HOME}/lib/ext folder of my Maven installation as well as in the folder of Jenkins, just to be sure.

But what I really don't understand is why it works in command line and not in Jenkins. I don't see where the difference is, as I'm using the same version of Maven and the same config file. I don't have enough knowledge of the platform resolution process to understand: where does Maven stop, where does Tycho stop and where does pure P2 code start in the platform resolution process, and how does the chain of transmission of credentials work? I thought that TYCHO-279 had solved the problem, but from the fix I was not able to completely understand the process.

To me it seems that Tycho should configure the proxy/credentials resolution in p2 before invoking it to resolve the platform, so the bug would be in the Tycho/Jenkins interaction, but I may be wrong.

To try the httpclient4, I have to consume the newest version of org.eclipse.ecf.filetransfer.feature, that's right? What would be the recommended way of doing (or is it what I have to ask to p2-***@eclipse.org)?

Vincent

Vincent Girard-Reydet - LNG Optimisation - BP Gas Marketing Limited, 20 Canada Square, London, E14 5NJ
t: +44 20 7948 5122
BP Gas Marketing Limited, a company registered in England and Wales with the company number 908982, VAT number GB 365 6789 95 and whose registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16 7BP
This message may contain information that is privileged or confidential. If you are not the intended recipient please delete it and inform the sender immediately. Within the bounds of law, the UK based entities in the Trading arm of the BP Group retain all emails and IMs and monitor them to ensure compliance with their internal policies and for other legitimate business purposes.


-----Original Message-----
From: tycho-user-***@eclipse.org [mailto:tycho-user-***@eclipse.org] On Behalf Of Sievers, Jan
Sent: 18 February 2013 16:28
To: Tycho user list
Subject: Re: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy

are you using maven-encrypted passwords?
https://bugs.eclipse.org/bugs/show_bug.cgi?id=394023

are you sure you added proxy credentials in settings.xml [1] ?

other than that you may have hit an NTLM limitation in the outdated apache httpclient 3.x still used by p2.
here is the bug to upgrade to httpclient v4 which seems to indicate it has better support for NTLM:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449#c2

in this case I suggest to ask on p2-***@eclipse.org for any hints.

Regards
Jan

[1] http://maven.apache.org/settings.html#Proxies



From: tycho-user-***@eclipse.org [mailto:tycho-user-***@eclipse.org] On Behalf Of Girard-reydet, Vincent
Sent: Montag, 18. Februar 2013 16:07
To: tycho-***@eclipse.org
Subject: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy

Hi folks,

I'm struggling to configure a Maven build using Tycho on Jenkins behind my company's proxy. I have a set of modules that build perfectly well from the console (I just rebuilt everything using a fresh m2 repo and the build is completely successful). Now I' trying to automate it with Jenkins (see the full env spec at the end of the email). I have configured a Maven build in Jenkins. For the local and global settings, I'm pointing at the exact same settings file I'm using in the console. This settings file contains the proxy definition, with the username, password and everything (and working fine in console). The build is using a private Maven repo.
For my build, I'm using a target platform file (NOT using p2 repositories in the pom.xml), as per the below. This platform is resolved OK when building in console.

The Jenkins build uses correctly the proxy information to download all Maven plugins (including Tycho plugins). However, when Tycho starts resolving the platform, it somehow fails to collect the proxy information (which it should since TYCHO-279 is now closed). I tried all the workarounds described to circumvene TYCHO-279 (using the ECF proxy properties), I tried also using a Nexus repo, nothing worked. I'm a bit screwed new, it seems to be somehow Jenkins-specific but I don't' have a clue what's going on.


Would someone be able to help me investigate the problem?

Environment spec:
- Windows 7
- NTLM2 proxy
- Maven 3.0.4
- JDK 1.6.39 x86_64
- Jenkins v1.501 (out of the box install, running as a Windows service)
- Tycho v0.16

Platform configuration in parent pom:

      <plugin>
        <groupId>org.eclipse.tycho</groupId>
        <artifactId>target-platform-configuration</artifactId>
        <version>${tycho.version}</version>
        <configuration>
         <target>
          <artifact>
             <groupId>FAKE_GROUP_ID</groupId>
             <artifactId>target</artifactId>
             <version>1.0.0</version>
             <classifier>indigo</classifier>
          </artifact>
         </target>
          <environments>
            <environment>
              <os>win32</os>
              <ws>win32</ws>
              <arch>x86</arch>
            </environment>
            <environment>
              <os>win32</os>
              <ws>win32</ws>
              <arch>x86_64</arch>
            </environment>
          </environments>
        </configuration>
      </plugin>

Platform definition :
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?pde version="3.6"?>

<target name="indigo" sequenceNumber="14"> <locations> <location includeAllPlatforms="false" includeMode="slicer" includeSource="false" type="InstallableUnit"> <unit id="org.eclipse.equinox.server.servletbridge.feature.group" version="1.0.100.v20110502-42DF9oB5865K5D"/>
<repository location="http://download.eclipse.org/eclipse/updates/3.7"/>
</location>
<location includeAllPlatforms="false" includeMode="slicer" includeSource="false" type="InstallableUnit"> <unit id="org.eclipse.ajdt.feature.group" version="2.2.0.e37x-RELEASE-20120704-0900"/>
<unit id="org.eclipse.equinox.weaving.sdk.feature.group" version="1.0.200.I20120427-0800"/>
<repository location="http://download.eclipse.org/tools/ajdt/37/update"/>
</location>
<location includeAllPlatforms="false" includeMode="slicer" includeSource="false" type="InstallableUnit"> <unit id="org.eclipse.sdk.ide" version="3.7.2.M20120208-0800"/> <unit id="org.eclipse.emf.ecore.feature.group" version="2.7.1.v20120127-1122"/> <unit id="org.eclipse.emf.sdk.feature.group" version="2.7.2.v20120130-0943"/> <unit id="org.eclipse.emf.compare.feature.group" version="1.2.2.v20120214-0915"/> <unit id="org.eclipse.emf.query.feature.group" version="1.5.0.v20110502-1600-218Z7w311A142A1142"/>
<unit id="org.eclipse.draw2d.feature.group" version="3.7.2.v20110927-2020-4617w3122212803131"/>
<unit id="org.eclipse.xsd.sdk.feature.group" version="2.7.1.v20120130-0943"/> <unit id="org.eclipse.objectteams.otequinox.feature.group" version="2.0.2.201202051448"/> <unit id="org.eclipse.equinox.sdk.feature.group" version="3.7.1.R37x_v20110907-7M7W8i8eNV4WsRkue-4Vq4J6pCyW"/>
<unit id="javax.transaction" version="1.1.1.v201105210645"/>

<unit id="org.eclipse.equinox.executable.feature.group" version="3.5.0.v20110530-7P7NFUFFLWUl76mart"/>
<unit id="org.eclipse.jdt.feature.group" version="3.7.1.r371_v20110810-0800-7z8gFcoFMLfTabvKsR5Qm9rBGEBK"/>
<unit id="org.eclipse.rcp.sdk.id" version="3.7.1.M20110909-1335"/>

<repository location="http://download.eclipse.org/releases/indigo"/>
</location>
</locations>
<environment>
<arch>x86</arch>
</environment>
</target>

Error log in Jenkins (no error when building from console):

Downloaded: http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plugin/1.3/aspectj-maven-plugin-1.3.jar (37 KB at 63.6 KB/sec) [INFO] Computing target platform for MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\jenkins_workspace\test\<MODULE_ID_HIDDEN>\pom.xml
18-Feb-2013 12:30:08 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 12:30:08 org.apache.commons.httpclient.HttpMethodDirector processProxyAuthChallenge
INFO: No credentials available for NTLM <any realm>@iaccess.bp.com:80 mavenExecutionResult exceptions not empty message : Internal error: java.lang.RuntimeException: Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target

cause : Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
Stack trace :
org.apache.maven.InternalErrorException: Internal error: java.lang.RuntimeException: Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
         at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:168)
         at org.jvnet.hudson.maven3.launcher.Maven3Launcher.main(Maven3Launcher.java:79)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:597)
         at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329)
         at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239)
         at org.jvnet.hudson.maven3.agent.Maven3Main.launch(Maven3Main.java:158)
         at hudson.maven.Maven3Builder.call(Maven3Builder.java:100)
         at hudson.maven.Maven3Builder.call(Maven3Builder.java:66)
         at hudson.remoting.UserRequest.perform(UserRequest.java:118)
         at hudson.remoting.UserRequest.perform(UserRequest.java:48)
         at hudson.remoting.Request$2.run(Request.java:326)
         at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
         at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.RuntimeException: Failed to resolve target definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
         at org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileContentToTargetPlatform(P2TargetPlatformResolver.java:338)
         at org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.computeTargetPlatform(P2TargetPlatformResolver.java:207)
         at org.eclipse.tycho.core.resolver.DefaultTychoDependencyResolver.resolveProject(DefaultTychoDependencyResolver.java:94)
         at org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.afterProjectsRead(TychoMavenLifecycleParticipant.java:82)
         at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:274)
         at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
         ... 19 more
Caused by: org.eclipse.tycho.p2.target.facade.TargetDefinitionResolutionException: Failed to load metadata repository from location http://download.eclipse.org/eclipse/updates/3.7
         at org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(TargetDefinitionResolver.java:190)
         at org.eclipse.tycho.p2.target.TargetDefinitionResolver.resolveContent(TargetDefinitionResolver.java:109)
         at org.eclipse.tycho.p2.target.TargetDefinitionResolverService.resolveFromArguments(TargetDefinitionResolverService.java:64)
         at org.eclipse.tycho.p2.target.TargetDefinitionResolverService.getTargetDefinitionContent(TargetDefinitionResolverService.java:54)
         at org.eclipse.tycho.p2.target.TargetPlatformBuilderImpl.addTargetDefinition(TargetPlatformBuilderImpl.java:289)
         at org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileContentToTargetPlatform(P2TargetPlatformResolver.java:331)
         ... 24 more
Caused by: org.eclipse.equinox.p2.core.ProvisionException: HTTP Proxy Authentication Required: http://download.eclipse.org/eclipse/updates/3.7/content.xml
         at org.eclipse.equinox.internal.p2.repository.CacheManager.createCache(CacheManager.java:192)
         at org.eclipse.tycho.p2.remote.TychoP2RepositoryCacheManager.createCache(TychoP2RepositoryCacheManager.java:62)
         at org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataRepositoryFactory.getLocalFile(SimpleMetadataRepositoryFactory.java:66)
         at org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataRepositoryFactory.load(SimpleMetadataRepositoryFactory.java:88)
         at org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.factoryLoad(MetadataRepositoryManager.java:57)
         at org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryManager.loadRepository(AbstractRepositoryManager.java:758)
         at org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryManager.loadRepository(AbstractRepositoryManager.java:651)
         at org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.loadRepository(MetadataRepositoryManager.java:96)
         at org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.loadRepository(MetadataRepositoryManager.java:92)
         at org.eclipse.tycho.p2.remote.RemoteMetadataRepositoryManager.loadRepository(RemoteMetadataRepositoryManager.java:82)
         at org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(TargetDefinitionResolver.java:188)
         ... 29 more
Caused by: org.eclipse.ecf.filetransfer.BrowseFileTransferException: Proxy auth required
         at org.eclipse.ecf.provider.filetransfer.browse.URLFileSystemBrowser.runRequest(URLFileSystemBrowser.java:129)
         at org.eclipse.ecf.provider.filetransfer.browse.AbstractFileSystemBrowser$DirectoryJob.run(AbstractFileSystemBrowser.java:69)
         at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)
channel stopped

Output from the console (OK):

Downloaded: http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plugin/1.3/aspectj-maven-plugin-1.3.jar (37 KB at 63.6 KB/sec) [INFO] Scanning for projects...
[INFO] Computing target platform for MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
18-Feb-2013 14:58:28 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 14:58:28 org.apache.commons.httpclient.HttpMethodDirector processProxyAuthChallenge
INFO: No credentials available for NTLM <any realm>@iaccess.bp.com:80 [WARNING] De-selecting bundles in a target definition file is not supported. See http://wiki.eclipse.org/Tycho_Messages_Explained#Target_File_Include_Bundles for alternatives.
[INFO] Resolving dependencies of MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
[WARNING] The following locally built units have been used to resolve project dependencies:
<list of dependencies to local modules>
[INFO] Resolving class path of MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
[WARNING] Dependency from C:\git\ok\<MODULE_ID_HIDDEN> to nested classpath entry C:\git\ok\<OTHER_MODULE>\icons can not be represented in Maven model and will not be visible to non-OSGi aware Maven plugins

Thank,
Vincent
Vincent Girard-Reydet - BP Gas Marketing Limited, 20 Canada Square, London, E14 5NJ BP Gas Marketing Limited, a company registered in England and Wales with the company number 908982, VAT number GB 365 6789 95 and whose registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16 7BP This message may contain information that is privileged or confidential. If you are not the intended recipient please delete it and inform the sender immediately. Within the bounds of law, the UK based entities in the Trading arm of the BP Group retain all emails and IMs and monitor them to ensure compliance with their internal policies and for other legitimate business purposes.
Girard-reydet, Vincent
2013-02-19 11:30:52 UTC
Permalink
Hi Jan,

Just to clarify things:
- everything happens on my machine (no other machine involved), which is
running Windows 7 (see full env spec in my original email)
- each time the same Maven settings file is used
- Using the same HTTP proxy configuration (I'm using the "native"
settings which are correctly mapped to the proxy to use) I am able to
download and install artifacts in Eclipse. In fact I created the target
file used by Tycho in my Eclipse IDE, it had no problem downloading the
update site contents
- Using the very same Maven settings WORKS OK when I build from my local
console, and FAILS when building inside a Jenkins instance running on my
machine and using my local settings

I have attached to this email two anonymized log files as you requested
in your other answer. The first one shows how it happens on the console,
when it's successful. The other is the Jenkins log file, where I tried
to define
-Dorg.eclipse.ecf.provider.filetransfer.excludeContributors=org.eclipse.
ecf.provider.filetransfer.httpclient as per your suggestion.
Both logs have been taken with maven -e -X.

Note that ${HOME}\.m2\settings.xml does not exist, I'm using only the
global config file at ${MAVEN_HOME}\settings.xml.

Vincent

Vincent Girard-Reydet - LNG Optimisation - BP Gas Marketing Limited, 20
Canada Square, London, E14 5NJ
t: +44 20 7948 5122
BP Gas Marketing Limited, a company registered in England and Wales with
the company number 908982, VAT number GB 365 6789 95 and whose
registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16
7BP
This message may contain information that is privileged or confidential.
If you are not the intended recipient please delete it and inform the
sender immediately. Within the bounds of law, the UK based entities in
the Trading arm of the BP Group retain all emails and IMs and monitor
them to ensure compliance with their internal policies and for other
legitimate business purposes.


-----Original Message-----
From: tycho-user-***@eclipse.org
[mailto:tycho-user-***@eclipse.org] On Behalf Of Sievers, Jan
Sent: 19 February 2013 08:45
To: P2 developer discussions; Tycho user list
Subject: Re: [tycho-user] [p2-dev] Tycho 0.16 + Jenkins + NTLM2 proxy

thanks Scott I think this will help to dig a little deeper.
I don't know how tycho handles the download...so I can't say exactly
what's happening here.
We delegate this to p2/ECF so it should work as if you would
install/update with your IDE.
All we do is configure the HTTP proxy via org.eclipse.core.net (and
register the default java.net.Authenticator, same as
org.eclipse.core.net.ui does IIRC).

two follow-up questions:

1. is there any log/trace so we can check the fallback to the JRE proxy
provider is actually happening in this case?
2. if the JRE proxy fallback happens, any chance different JDKs
(OS/version) may have differing/no support for NTLM proxy
authentication?

@Vincent another thing you can check to help us pin down the issue:
using the same HTTP proxy configurations in your IDE (in Window>Prefs),
can you successfully update/install any bundles e.g. from eclipse.org ?
just to make sure I got this right: you say the build works fine on one
machine but doesn't on another?
what's the difference between those 2 machines (OS, JDK) ?

If the httpclient v4 support is available but not consumed yet,
ultimately we may decide to bundle httpclient v4 with tycho if that
solves the issue.

Regards,
Jan


-----Original Message-----
From: p2-dev-***@eclipse.org [mailto:p2-dev-***@eclipse.org] On
Behalf Of Scott Lewis
Sent: Montag, 18. Februar 2013 23:50
To: p2-***@eclipse.org
Subject: Re: [p2-dev] Tycho 0.16 + Jenkins + NTLM2 proxy

FYI,
- cross-posting to p2-dev -
tycho takes the maven HTTP proxy configuration from settings.xml and
configures p2 accordingly (via proxyservice API in
org.eclipse.core.net) if resolution and download of maven plugins
works but p2 resolution does not, this is a hint that either
a) tycho fails to transfer the proxy info to p2 or
b) p2 / org.eclipse.core.net does not support NTLM proxy
authentication
since nobody reported a) yet I doubt this and suspect b)
I don't know how tycho handles the download...so I can't say exactly
what's happening here.

WRT b...p2 (via ECF filetransfer) does indeed support NTLMv2. There's a
long history to this...see [1]...but the upshot is that ECF filetransfer
has a workaround to not use the apache httpclient 3.1 provider, in the
event of NTLMv2 (since apache httpclient < 4.0 did not support NTLMv2,
and that's what ECF used until recently).
@p2-devs any hints as to b) above?
https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449#c2 may be
related.
@Vincent just in case using apache httpclient v4 would help, this is
not up to you; it's a change which would have to be done in p2 (or at
least in tycho)
No...it's a change that's underway...intended for Kepler: i.e. see this
enhancement [2]. The desire here is that apache httpclient 4.x
reportedly has support for ntlmv2 proxies, where httpclient 3.1 did
not. We have a completed provider based upon httpclient 4.x, and would

appreciate some help with testing against various proxy configurations
in coming integration builds.

But again...since I don't know how/what tycho does WRT downloading, this
might not address your proxy issue.

Thanks,

Scott

[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=252002
[2] https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449
Another thing you should try is to enable advanced HTTP tracing as per
[1] and post the traces
Regards,
Jan
[1] http://wiki.eclipse.org/Equinox_p2_Reporting_Problems
-----Original Message-----
Vincent
Sent: Montag, 18. Februar 2013 18:50
To: Tycho user list
Subject: Re: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy
Hi Jan,
Thank you for your answer. No I'm not using encrypted passwords for
now. I have triple-checked the settings and it's using the proxy (I've
tried removing the section completely and the Jenkins fails to download
the Maven plugins). Here is the proxy section I'm using (passwords,
<proxies>
<proxy>
<id>bp</id>
<active>true</active>
<protocol>http</protocol>
<username>MYUSERNAME</username>
<password>MYPASSWORD</password>
<host>my.proxy.host</host>
<port>808080</port>
<!--<nonProxyHosts>local.net|some.host.com</nonProxyHosts>-->
</proxy>
</proxies>
Note that I have dropped the wagon-http-lightweight-2.2.jar in the
${MAVEN_HOME}/lib/ext folder of my Maven installation as well as in the
folder of Jenkins, just to be sure.
But what I really don't understand is why it works in command line and
not in Jenkins. I don't see where the difference is, as I'm using the
same version of Maven and the same config file. I don't have enough
knowledge of the platform resolution process to understand: where does
Maven stop, where does Tycho stop and where does pure P2 code start in
the platform resolution process, and how does the chain of transmission
of credentials work? I thought that TYCHO-279 had solved the problem,
but from the fix I was not able to completely understand the process.
To me it seems that Tycho should configure the proxy/credentials
resolution in p2 before invoking it to resolve the platform, so the bug
would be in the Tycho/Jenkins interaction, but I may be wrong.
To try the httpclient4, I have to consume the newest version of
org.eclipse.ecf.filetransfer.feature, that's right? What would be the
recommended way of doing (or is it what I have to ask to
Vincent
Vincent Girard-Reydet - LNG Optimisation - BP Gas Marketing Limited,
20 Canada Square, London, E14 5NJ
t: +44 20 7948 5122
BP Gas Marketing Limited, a company registered in England and Wales
with the company number 908982, VAT number GB 365 6789 95 and whose
registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16
7BP This message may contain information that is privileged or
confidential. If you are not the intended recipient please delete it and
inform the sender immediately. Within the bounds of law, the UK based
entities in the Trading arm of the BP Group retain all emails and IMs
and monitor them to ensure compliance with their internal policies and
for other legitimate business purposes.
-----Original Message-----
Sent: 18 February 2013 16:28
To: Tycho user list
Subject: Re: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy
are you using maven-encrypted passwords?
https://bugs.eclipse.org/bugs/show_bug.cgi?id=394023
are you sure you added proxy credentials in settings.xml [1] ?
other than that you may have hit an NTLM limitation in the outdated
apache httpclient 3.x still used by p2.
here is the bug to upgrade to httpclient v4 which seems to indicate
https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449#c2
Regards
Jan
[1] http://maven.apache.org/settings.html#Proxies
Vincent
Sent: Montag, 18. Februar 2013 16:07
Subject: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy
Hi folks,
I'm struggling to configure a Maven build using Tycho on Jenkins
behind my company's proxy. I have a set of modules that build perfectly
well from the console (I just rebuilt everything using a fresh m2 repo
and the build is completely successful). Now I' trying to automate it
with Jenkins (see the full env spec at the end of the email). I have
configured a Maven build in Jenkins. For the local and global settings,
I'm pointing at the exact same settings file I'm using in the console.
This settings file contains the proxy definition, with the username,
password and everything (and working fine in console). The build is
using a private Maven repo.
For my build, I'm using a target platform file (NOT using p2
repositories in the pom.xml), as per the below. This platform is
resolved OK when building in console.
The Jenkins build uses correctly the proxy information to download all
Maven plugins (including Tycho plugins). However, when Tycho starts
resolving the platform, it somehow fails to collect the proxy
information (which it should since TYCHO-279 is now closed). I tried all
the workarounds described to circumvene TYCHO-279 (using the ECF proxy
properties), I tried also using a Nexus repo, nothing worked. I'm a bit
screwed new, it seems to be somehow Jenkins-specific but I don't' have a
clue what's going on.
Would someone be able to help me investigate the problem?
- Windows 7
- NTLM2 proxy
- Maven 3.0.4
- JDK 1.6.39 x86_64
- Jenkins v1.501 (out of the box install, running as a Windows
service)
- Tycho v0.16
<plugin>
<groupId>org.eclipse.tycho</groupId>
<artifactId>target-platform-configuration</artifactId>
<version>${tycho.version}</version>
<configuration>
<target>
<artifact>
<groupId>FAKE_GROUP_ID</groupId>
<artifactId>target</artifactId>
<version>1.0.0</version>
<classifier>indigo</classifier>
</artifact>
</target>
<environments>
<environment>
<os>win32</os>
<ws>win32</ws>
<arch>x86</arch>
</environment>
<environment>
<os>win32</os>
<ws>win32</ws>
<arch>x86_64</arch>
</environment>
</environments>
</configuration>
</plugin>
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?pde
version="3.6"?>
<target name="indigo" sequenceNumber="14"> <locations> <location
includeAllPlatforms="false" includeMode="slicer" includeSource="false"
type="InstallableUnit"> <unit
id="org.eclipse.equinox.server.servletbridge.feature.group"
version="1.0.100.v20110502-42DF9oB5865K5D"/>
<repository
location="http://download.eclipse.org/eclipse/updates/3.7"/>
</location>
<location includeAllPlatforms="false" includeMode="slicer"
includeSource="false" type="InstallableUnit"> <unit
id="org.eclipse.ajdt.feature.group"
version="2.2.0.e37x-RELEASE-20120704-0900"/>
<unit id="org.eclipse.equinox.weaving.sdk.feature.group"
version="1.0.200.I20120427-0800"/>
<repository
location="http://download.eclipse.org/tools/ajdt/37/update"/>
</location>
<location includeAllPlatforms="false" includeMode="slicer"
includeSource="false" type="InstallableUnit"> <unit
id="org.eclipse.sdk.ide" version="3.7.2.M20120208-0800"/> <unit
id="org.eclipse.emf.ecore.feature.group"
version="2.7.1.v20120127-1122"/> <unit
id="org.eclipse.emf.sdk.feature.group"
version="2.7.2.v20120130-0943"/> <unit
id="org.eclipse.emf.compare.feature.group"
version="1.2.2.v20120214-0915"/> <unit
id="org.eclipse.emf.query.feature.group"
version="1.5.0.v20110502-1600-218Z7w311A142A1142"/>
<unit id="org.eclipse.draw2d.feature.group"
version="3.7.2.v20110927-2020-4617w3122212803131"/>
<unit id="org.eclipse.xsd.sdk.feature.group"
version="2.7.1.v20120130-0943"/> <unit
id="org.eclipse.objectteams.otequinox.feature.group"
version="2.0.2.201202051448"/> <unit
id="org.eclipse.equinox.sdk.feature.group"
version="3.7.1.R37x_v20110907-7M7W8i8eNV4WsRkue-4Vq4J6pCyW"/>
<unit id="javax.transaction" version="1.1.1.v201105210645"/>
<unit id="org.eclipse.equinox.executable.feature.group"
version="3.5.0.v20110530-7P7NFUFFLWUl76mart"/>
<unit id="org.eclipse.jdt.feature.group"
version="3.7.1.r371_v20110810-0800-7z8gFcoFMLfTabvKsR5Qm9rBGEBK"/>
<unit id="org.eclipse.rcp.sdk.id" version="3.7.1.M20110909-1335"/>
<repository location="http://download.eclipse.org/releases/indigo"/>
</location>
</locations>
<environment>
<arch>x86</arch>
</environment>
</target>
http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-pl
ugin/1.3/aspectj-maven-plugin-1.3.jar (37 KB at 63.6 KB/sec) [INFO]
C:\jenkins_workspace\test\<MODULE_ID_HIDDEN>\pom.xml
18-Feb-2013 12:30:08
org.apache.commons.httpclient.auth.AuthChallengeProcessor
selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 12:30:08 org.apache.commons.httpclient.HttpMethodDirector
processProxyAuthChallenge
java.lang.RuntimeException: Failed to resolve target definition
C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
cause : Failed to resolve target definition
C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
java.lang.RuntimeException: Failed to resolve target definition
C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
at
org.apache.maven.DefaultMaven.execute(DefaultMaven.java:168)
at
org.jvnet.hudson.maven3.launcher.Maven3Launcher.main(Maven3Launcher.java
:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launche
r.java:329)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:2
39)
at
org.jvnet.hudson.maven3.agent.Maven3Main.launch(Maven3Main.java:158)
at hudson.maven.Maven3Builder.call(Maven3Builder.java:100)
at hudson.maven.Maven3Builder.call(Maven3Builder.java:66)
at hudson.remoting.UserRequest.perform(UserRequest.java:118)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:326)
at
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorS
ervice.java:72)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecuto
r.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
va:918)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.RuntimeException: Failed to resolve target
definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
at
org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileCont
entToTargetPlatform(P2TargetPlatformResolver.java:338)
at
org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.computeTargetPlat
form(P2TargetPlatformResolver.java:207)
at
org.eclipse.tycho.core.resolver.DefaultTychoDependencyResolver.resolvePr
oject(DefaultTychoDependencyResolver.java:94)
at
org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.afterProject
sRead(TychoMavenLifecycleParticipant.java:82)
at
org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:274)
at
org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
... 19 more
org.eclipse.tycho.p2.target.facade.TargetDefinitionResolutionException:
Failed to load metadata repository from location
http://download.eclipse.org/eclipse/updates/3.7
at
org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(Targ
etDefinitionResolver.java:190)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolver.resolveContent(Targ
etDefinitionResolver.java:109)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolverService.resolveFromA
rguments(TargetDefinitionResolverService.java:64)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolverService.getTargetDef
initionContent(TargetDefinitionResolverService.java:54)
at
org.eclipse.tycho.p2.target.TargetPlatformBuilderImpl.addTargetDefinitio
n(TargetPlatformBuilderImpl.java:289)
at
org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileCont
entToTargetPlatform(P2TargetPlatformResolver.java:331)
... 24 more
Caused by: org.eclipse.equinox.p2.core.ProvisionException: HTTP Proxy
Authentication Required:
http://download.eclipse.org/eclipse/updates/3.7/content.xml
at
org.eclipse.equinox.internal.p2.repository.CacheManager.createCache(Cach
eManager.java:192)
at
org.eclipse.tycho.p2.remote.TychoP2RepositoryCacheManager.createCache(Ty
choP2RepositoryCacheManager.java:62)
at
org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataReposi
toryFactory.getLocalFile(SimpleMetadataRepositoryFactory.java:66)
at
org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataReposi
toryFactory.load(SimpleMetadataRepositoryFactory.java:88)
at
org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryMa
nager.factoryLoad(MetadataRepositoryManager.java:57)
at
org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryMan
ager.loadRepository(AbstractRepositoryManager.java:758)
at
org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryMan
ager.loadRepository(AbstractRepositoryManager.java:651)
at
org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryMa
nager.loadRepository(MetadataRepositoryManager.java:96)
at
org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryMa
nager.loadRepository(MetadataRepositoryManager.java:92)
at
org.eclipse.tycho.p2.remote.RemoteMetadataRepositoryManager.loadReposito
ry(RemoteMetadataRepositoryManager.java:82)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(Targ
etDefinitionResolver.java:188)
... 29 more
Proxy auth required
at
org.eclipse.ecf.provider.filetransfer.browse.URLFileSystemBrowser.runReq
uest(URLFileSystemBrowser.java:129)
at
org.eclipse.ecf.provider.filetransfer.browse.AbstractFileSystemBrowser$D
irectoryJob.run(AbstractFileSystemBrowser.java:69)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)
channel stopped
http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plug
in/1.3/aspectj-maven-plugin-1.3.jar (37 KB at 63.6 KB/sec) [INFO]
Scanning for projects...
C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
18-Feb-2013 14:58:28
org.apache.commons.httpclient.auth.AuthChallengeProcessor
selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 14:58:28 org.apache.commons.httpclient.HttpMethodDirector
processProxyAuthChallenge
[WARNING] De-selecting bundles in a target definition file is not
supported. See
http://wiki.eclipse.org/Tycho_Messages_Explained#Target_File_Include_Bun
dles for alternatives.
C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
[WARNING] The following locally built units have been used to resolve
<list of dependencies to local modules> [INFO] Resolving class path of
C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
[WARNING] Dependency from C:\git\ok\<MODULE_ID_HIDDEN> to nested
classpath entry C:\git\ok\<OTHER_MODULE>\icons can not be represented
in Maven model and will not be visible to non-OSGi aware Maven plugins
Thank,
Vincent
Vincent Girard-Reydet - BP Gas Marketing Limited, 20 Canada Square,
London, E14 5NJ BP Gas Marketing Limited, a company registered in
England and Wales with the company number 908982, VAT number GB 365 6789
95 and whose registered office is Chertsey Road, Sunbury on Thames,
Middlesex, TW16 7BP This message may contain information that is
privileged or confidential. If you are not the intended recipient please
delete it and inform the sender immediately. Within the bounds of law,
the UK based entities in the Trading arm of the BP Group retain all
emails and IMs and monitor them to ensure compliance with their internal
policies and for other legitimate business purposes.
_______________________________________________
tycho-user mailing list
https://dev.eclipse.org/mailman/listinfo/tycho-user
_______________________________________________
tycho-user mailing list
https://dev.eclipse.org/mailman/listinfo/tycho-user
_______________________________________________
p2-dev mailing list
https://dev.eclipse.org/mailman/listinfo/p2-dev
Sievers, Jan
2013-02-19 12:56:23 UTC
Permalink
hm. instead of the maven3 jenkins integration, can you use a jenkins freestyle job and start mvn in a console build step instead ?
just to make sure you do exactly the same thing as on the console where it works.

Jan

-----Original Message-----
From: tycho-user-***@eclipse.org [mailto:tycho-user-***@eclipse.org] On Behalf Of Girard-reydet, Vincent
Sent: Dienstag, 19. Februar 2013 12:31
To: Tycho user list; P2 developer discussions
Subject: Re: [tycho-user] [p2-dev] Tycho 0.16 + Jenkins + NTLM2 proxy

Hi Jan,

Just to clarify things:
- everything happens on my machine (no other machine involved), which is
running Windows 7 (see full env spec in my original email)
- each time the same Maven settings file is used
- Using the same HTTP proxy configuration (I'm using the "native"
settings which are correctly mapped to the proxy to use) I am able to
download and install artifacts in Eclipse. In fact I created the target
file used by Tycho in my Eclipse IDE, it had no problem downloading the
update site contents
- Using the very same Maven settings WORKS OK when I build from my local
console, and FAILS when building inside a Jenkins instance running on my
machine and using my local settings

I have attached to this email two anonymized log files as you requested
in your other answer. The first one shows how it happens on the console,
when it's successful. The other is the Jenkins log file, where I tried
to define
-Dorg.eclipse.ecf.provider.filetransfer.excludeContributors=org.eclipse.
ecf.provider.filetransfer.httpclient as per your suggestion.
Both logs have been taken with maven -e -X.

Note that ${HOME}\.m2\settings.xml does not exist, I'm using only the
global config file at ${MAVEN_HOME}\settings.xml.

Vincent

Vincent Girard-Reydet - LNG Optimisation - BP Gas Marketing Limited, 20
Canada Square, London, E14 5NJ
t: +44 20 7948 5122
BP Gas Marketing Limited, a company registered in England and Wales with
the company number 908982, VAT number GB 365 6789 95 and whose
registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16
7BP
This message may contain information that is privileged or confidential.
If you are not the intended recipient please delete it and inform the
sender immediately. Within the bounds of law, the UK based entities in
the Trading arm of the BP Group retain all emails and IMs and monitor
them to ensure compliance with their internal policies and for other
legitimate business purposes.


-----Original Message-----
From: tycho-user-***@eclipse.org
[mailto:tycho-user-***@eclipse.org] On Behalf Of Sievers, Jan
Sent: 19 February 2013 08:45
To: P2 developer discussions; Tycho user list
Subject: Re: [tycho-user] [p2-dev] Tycho 0.16 + Jenkins + NTLM2 proxy

thanks Scott I think this will help to dig a little deeper.
I don't know how tycho handles the download...so I can't say exactly
what's happening here.
We delegate this to p2/ECF so it should work as if you would
install/update with your IDE.
All we do is configure the HTTP proxy via org.eclipse.core.net (and
register the default java.net.Authenticator, same as
org.eclipse.core.net.ui does IIRC).

two follow-up questions:

1. is there any log/trace so we can check the fallback to the JRE proxy
provider is actually happening in this case?
2. if the JRE proxy fallback happens, any chance different JDKs
(OS/version) may have differing/no support for NTLM proxy
authentication?

@Vincent another thing you can check to help us pin down the issue:
using the same HTTP proxy configurations in your IDE (in Window>Prefs),
can you successfully update/install any bundles e.g. from eclipse.org ?
just to make sure I got this right: you say the build works fine on one
machine but doesn't on another?
what's the difference between those 2 machines (OS, JDK) ?

If the httpclient v4 support is available but not consumed yet,
ultimately we may decide to bundle httpclient v4 with tycho if that
solves the issue.

Regards,
Jan


-----Original Message-----
From: p2-dev-***@eclipse.org [mailto:p2-dev-***@eclipse.org] On
Behalf Of Scott Lewis
Sent: Montag, 18. Februar 2013 23:50
To: p2-***@eclipse.org
Subject: Re: [p2-dev] Tycho 0.16 + Jenkins + NTLM2 proxy

FYI,
- cross-posting to p2-dev -
tycho takes the maven HTTP proxy configuration from settings.xml and
configures p2 accordingly (via proxyservice API in
org.eclipse.core.net) if resolution and download of maven plugins
works but p2 resolution does not, this is a hint that either
a) tycho fails to transfer the proxy info to p2 or
b) p2 / org.eclipse.core.net does not support NTLM proxy
authentication
since nobody reported a) yet I doubt this and suspect b)
I don't know how tycho handles the download...so I can't say exactly
what's happening here.

WRT b...p2 (via ECF filetransfer) does indeed support NTLMv2. There's a
long history to this...see [1]...but the upshot is that ECF filetransfer
has a workaround to not use the apache httpclient 3.1 provider, in the
event of NTLMv2 (since apache httpclient < 4.0 did not support NTLMv2,
and that's what ECF used until recently).
@p2-devs any hints as to b) above?
https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449#c2 may be
related.
@Vincent just in case using apache httpclient v4 would help, this is
not up to you; it's a change which would have to be done in p2 (or at
least in tycho)
No...it's a change that's underway...intended for Kepler: i.e. see this
enhancement [2]. The desire here is that apache httpclient 4.x
reportedly has support for ntlmv2 proxies, where httpclient 3.1 did
not. We have a completed provider based upon httpclient 4.x, and would

appreciate some help with testing against various proxy configurations
in coming integration builds.

But again...since I don't know how/what tycho does WRT downloading, this
might not address your proxy issue.

Thanks,

Scott

[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=252002
[2] https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449
Another thing you should try is to enable advanced HTTP tracing as per
[1] and post the traces
Regards,
Jan
[1] http://wiki.eclipse.org/Equinox_p2_Reporting_Problems
-----Original Message-----
Vincent
Sent: Montag, 18. Februar 2013 18:50
To: Tycho user list
Subject: Re: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy
Hi Jan,
Thank you for your answer. No I'm not using encrypted passwords for
now. I have triple-checked the settings and it's using the proxy (I've
tried removing the section completely and the Jenkins fails to download
the Maven plugins). Here is the proxy section I'm using (passwords,
<proxies>
<proxy>
<id>bp</id>
<active>true</active>
<protocol>http</protocol>
<username>MYUSERNAME</username>
<password>MYPASSWORD</password>
<host>my.proxy.host</host>
<port>808080</port>
<!--<nonProxyHosts>local.net|some.host.com</nonProxyHosts>-->
</proxy>
</proxies>
Note that I have dropped the wagon-http-lightweight-2.2.jar in the
${MAVEN_HOME}/lib/ext folder of my Maven installation as well as in the
folder of Jenkins, just to be sure.
But what I really don't understand is why it works in command line and
not in Jenkins. I don't see where the difference is, as I'm using the
same version of Maven and the same config file. I don't have enough
knowledge of the platform resolution process to understand: where does
Maven stop, where does Tycho stop and where does pure P2 code start in
the platform resolution process, and how does the chain of transmission
of credentials work? I thought that TYCHO-279 had solved the problem,
but from the fix I was not able to completely understand the process.
To me it seems that Tycho should configure the proxy/credentials
resolution in p2 before invoking it to resolve the platform, so the bug
would be in the Tycho/Jenkins interaction, but I may be wrong.
To try the httpclient4, I have to consume the newest version of
org.eclipse.ecf.filetransfer.feature, that's right? What would be the
recommended way of doing (or is it what I have to ask to
Vincent
Vincent Girard-Reydet - LNG Optimisation - BP Gas Marketing Limited,
20 Canada Square, London, E14 5NJ
t: +44 20 7948 5122
BP Gas Marketing Limited, a company registered in England and Wales
with the company number 908982, VAT number GB 365 6789 95 and whose
registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16
7BP This message may contain information that is privileged or
confidential. If you are not the intended recipient please delete it and
inform the sender immediately. Within the bounds of law, the UK based
entities in the Trading arm of the BP Group retain all emails and IMs
and monitor them to ensure compliance with their internal policies and
for other legitimate business purposes.
-----Original Message-----
Sent: 18 February 2013 16:28
To: Tycho user list
Subject: Re: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy
are you using maven-encrypted passwords?
https://bugs.eclipse.org/bugs/show_bug.cgi?id=394023
are you sure you added proxy credentials in settings.xml [1] ?
other than that you may have hit an NTLM limitation in the outdated
apache httpclient 3.x still used by p2.
here is the bug to upgrade to httpclient v4 which seems to indicate
https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449#c2
Regards
Jan
[1] http://maven.apache.org/settings.html#Proxies
Vincent
Sent: Montag, 18. Februar 2013 16:07
Subject: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy
Hi folks,
I'm struggling to configure a Maven build using Tycho on Jenkins
behind my company's proxy. I have a set of modules that build perfectly
well from the console (I just rebuilt everything using a fresh m2 repo
and the build is completely successful). Now I' trying to automate it
with Jenkins (see the full env spec at the end of the email). I have
configured a Maven build in Jenkins. For the local and global settings,
I'm pointing at the exact same settings file I'm using in the console.
This settings file contains the proxy definition, with the username,
password and everything (and working fine in console). The build is
using a private Maven repo.
For my build, I'm using a target platform file (NOT using p2
repositories in the pom.xml), as per the below. This platform is
resolved OK when building in console.
The Jenkins build uses correctly the proxy information to download all
Maven plugins (including Tycho plugins). However, when Tycho starts
resolving the platform, it somehow fails to collect the proxy
information (which it should since TYCHO-279 is now closed). I tried all
the workarounds described to circumvene TYCHO-279 (using the ECF proxy
properties), I tried also using a Nexus repo, nothing worked. I'm a bit
screwed new, it seems to be somehow Jenkins-specific but I don't' have a
clue what's going on.
Would someone be able to help me investigate the problem?
- Windows 7
- NTLM2 proxy
- Maven 3.0.4
- JDK 1.6.39 x86_64
- Jenkins v1.501 (out of the box install, running as a Windows
service)
- Tycho v0.16
<plugin>
<groupId>org.eclipse.tycho</groupId>
<artifactId>target-platform-configuration</artifactId>
<version>${tycho.version}</version>
<configuration>
<target>
<artifact>
<groupId>FAKE_GROUP_ID</groupId>
<artifactId>target</artifactId>
<version>1.0.0</version>
<classifier>indigo</classifier>
</artifact>
</target>
<environments>
<environment>
<os>win32</os>
<ws>win32</ws>
<arch>x86</arch>
</environment>
<environment>
<os>win32</os>
<ws>win32</ws>
<arch>x86_64</arch>
</environment>
</environments>
</configuration>
</plugin>
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?pde
version="3.6"?>
<target name="indigo" sequenceNumber="14"> <locations> <location
includeAllPlatforms="false" includeMode="slicer" includeSource="false"
type="InstallableUnit"> <unit
id="org.eclipse.equinox.server.servletbridge.feature.group"
version="1.0.100.v20110502-42DF9oB5865K5D"/>
<repository
location="http://download.eclipse.org/eclipse/updates/3.7"/>
</location>
<location includeAllPlatforms="false" includeMode="slicer"
includeSource="false" type="InstallableUnit"> <unit
id="org.eclipse.ajdt.feature.group"
version="2.2.0.e37x-RELEASE-20120704-0900"/>
<unit id="org.eclipse.equinox.weaving.sdk.feature.group"
version="1.0.200.I20120427-0800"/>
<repository
location="http://download.eclipse.org/tools/ajdt/37/update"/>
</location>
<location includeAllPlatforms="false" includeMode="slicer"
includeSource="false" type="InstallableUnit"> <unit
id="org.eclipse.sdk.ide" version="3.7.2.M20120208-0800"/> <unit
id="org.eclipse.emf.ecore.feature.group"
version="2.7.1.v20120127-1122"/> <unit
id="org.eclipse.emf.sdk.feature.group"
version="2.7.2.v20120130-0943"/> <unit
id="org.eclipse.emf.compare.feature.group"
version="1.2.2.v20120214-0915"/> <unit
id="org.eclipse.emf.query.feature.group"
version="1.5.0.v20110502-1600-218Z7w311A142A1142"/>
<unit id="org.eclipse.draw2d.feature.group"
version="3.7.2.v20110927-2020-4617w3122212803131"/>
<unit id="org.eclipse.xsd.sdk.feature.group"
version="2.7.1.v20120130-0943"/> <unit
id="org.eclipse.objectteams.otequinox.feature.group"
version="2.0.2.201202051448"/> <unit
id="org.eclipse.equinox.sdk.feature.group"
version="3.7.1.R37x_v20110907-7M7W8i8eNV4WsRkue-4Vq4J6pCyW"/>
<unit id="javax.transaction" version="1.1.1.v201105210645"/>
<unit id="org.eclipse.equinox.executable.feature.group"
version="3.5.0.v20110530-7P7NFUFFLWUl76mart"/>
<unit id="org.eclipse.jdt.feature.group"
version="3.7.1.r371_v20110810-0800-7z8gFcoFMLfTabvKsR5Qm9rBGEBK"/>
<unit id="org.eclipse.rcp.sdk.id" version="3.7.1.M20110909-1335"/>
<repository location="http://download.eclipse.org/releases/indigo"/>
</location>
</locations>
<environment>
<arch>x86</arch>
</environment>
</target>
http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-pl
ugin/1.3/aspectj-maven-plugin-1.3.jar (37 KB at 63.6 KB/sec) [INFO]
C:\jenkins_workspace\test\<MODULE_ID_HIDDEN>\pom.xml
18-Feb-2013 12:30:08
org.apache.commons.httpclient.auth.AuthChallengeProcessor
selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 12:30:08 org.apache.commons.httpclient.HttpMethodDirector
processProxyAuthChallenge
java.lang.RuntimeException: Failed to resolve target definition
C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
cause : Failed to resolve target definition
C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
java.lang.RuntimeException: Failed to resolve target definition
C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
at
org.apache.maven.DefaultMaven.execute(DefaultMaven.java:168)
at
org.jvnet.hudson.maven3.launcher.Maven3Launcher.main(Maven3Launcher.java
:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launche
r.java:329)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:2
39)
at
org.jvnet.hudson.maven3.agent.Maven3Main.launch(Maven3Main.java:158)
at hudson.maven.Maven3Builder.call(Maven3Builder.java:100)
at hudson.maven.Maven3Builder.call(Maven3Builder.java:66)
at hudson.remoting.UserRequest.perform(UserRequest.java:118)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:326)
at
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorS
ervice.java:72)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecuto
r.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
va:918)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.RuntimeException: Failed to resolve target
definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
at
org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileCont
entToTargetPlatform(P2TargetPlatformResolver.java:338)
at
org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.computeTargetPlat
form(P2TargetPlatformResolver.java:207)
at
org.eclipse.tycho.core.resolver.DefaultTychoDependencyResolver.resolvePr
oject(DefaultTychoDependencyResolver.java:94)
at
org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.afterProject
sRead(TychoMavenLifecycleParticipant.java:82)
at
org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:274)
at
org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
... 19 more
org.eclipse.tycho.p2.target.facade.TargetDefinitionResolutionException:
Failed to load metadata repository from location
http://download.eclipse.org/eclipse/updates/3.7
at
org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(Targ
etDefinitionResolver.java:190)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolver.resolveContent(Targ
etDefinitionResolver.java:109)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolverService.resolveFromA
rguments(TargetDefinitionResolverService.java:64)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolverService.getTargetDef
initionContent(TargetDefinitionResolverService.java:54)
at
org.eclipse.tycho.p2.target.TargetPlatformBuilderImpl.addTargetDefinitio
n(TargetPlatformBuilderImpl.java:289)
at
org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileCont
entToTargetPlatform(P2TargetPlatformResolver.java:331)
... 24 more
Caused by: org.eclipse.equinox.p2.core.ProvisionException: HTTP Proxy
Authentication Required:
http://download.eclipse.org/eclipse/updates/3.7/content.xml
at
org.eclipse.equinox.internal.p2.repository.CacheManager.createCache(Cach
eManager.java:192)
at
org.eclipse.tycho.p2.remote.TychoP2RepositoryCacheManager.createCache(Ty
choP2RepositoryCacheManager.java:62)
at
org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataReposi
toryFactory.getLocalFile(SimpleMetadataRepositoryFactory.java:66)
at
org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataReposi
toryFactory.load(SimpleMetadataRepositoryFactory.java:88)
at
org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryMa
nager.factoryLoad(MetadataRepositoryManager.java:57)
at
org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryMan
ager.loadRepository(AbstractRepositoryManager.java:758)
at
org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryMan
ager.loadRepository(AbstractRepositoryManager.java:651)
at
org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryMa
nager.loadRepository(MetadataRepositoryManager.java:96)
at
org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryMa
nager.loadRepository(MetadataRepositoryManager.java:92)
at
org.eclipse.tycho.p2.remote.RemoteMetadataRepositoryManager.loadReposito
ry(RemoteMetadataRepositoryManager.java:82)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(Targ
etDefinitionResolver.java:188)
... 29 more
Proxy auth required
at
org.eclipse.ecf.provider.filetransfer.browse.URLFileSystemBrowser.runReq
uest(URLFileSystemBrowser.java:129)
at
org.eclipse.ecf.provider.filetransfer.browse.AbstractFileSystemBrowser$D
irectoryJob.run(AbstractFileSystemBrowser.java:69)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)
channel stopped
http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plug
in/1.3/aspectj-maven-plugin-1.3.jar (37 KB at 63.6 KB/sec) [INFO]
Scanning for projects...
C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
18-Feb-2013 14:58:28
org.apache.commons.httpclient.auth.AuthChallengeProcessor
selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 14:58:28 org.apache.commons.httpclient.HttpMethodDirector
processProxyAuthChallenge
[WARNING] De-selecting bundles in a target definition file is not
supported. See
http://wiki.eclipse.org/Tycho_Messages_Explained#Target_File_Include_Bun
dles for alternatives.
C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
[WARNING] The following locally built units have been used to resolve
<list of dependencies to local modules> [INFO] Resolving class path of
C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
[WARNING] Dependency from C:\git\ok\<MODULE_ID_HIDDEN> to nested
classpath entry C:\git\ok\<OTHER_MODULE>\icons can not be represented
in Maven model and will not be visible to non-OSGi aware Maven plugins
Thank,
Vincent
Vincent Girard-Reydet - BP Gas Marketing Limited, 20 Canada Square,
London, E14 5NJ BP Gas Marketing Limited, a company registered in
England and Wales with the company number 908982, VAT number GB 365 6789
95 and whose registered office is Chertsey Road, Sunbury on Thames,
Middlesex, TW16 7BP This message may contain information that is
privileged or confidential. If you are not the intended recipient please
delete it and inform the sender immediately. Within the bounds of law,
the UK based entities in the Trading arm of the BP Group retain all
emails and IMs and monitor them to ensure compliance with their internal
policies and for other legitimate business purposes.
_______________________________________________
tycho-user mailing list
https://dev.eclipse.org/mailman/listinfo/tycho-user
_______________________________________________
tycho-user mailing list
https://dev.eclipse.org/mailman/listinfo/tycho-user
_______________________________________________
p2-dev mailing list
https://dev.eclipse.org/mailman/listinfo/p2-dev
Girard-reydet, Vincent
2013-02-19 14:18:17 UTC
Permalink
Hi Jan,

Good call :-) I tried as you suggested (even going to the exact same
folder that works using the console build) and it didn't work either. I
think this must be environmental e.g. the proxy not allowing Jenkins to
authenticate to prevent attacks, or some weirdness like that. That
probably eliminates a pure Tycho or p2 issue.

Vincent

Vincent Girard-Reydet - LNG Optimisation - BP Gas Marketing Limited, 20
Canada Square, London, E14 5NJ
t: +44 20 7948 5122
BP Gas Marketing Limited, a company registered in England and Wales with
the company number 908982, VAT number GB 365 6789 95 and whose
registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16
7BP
This message may contain information that is privileged or confidential.
If you are not the intended recipient please delete it and inform the
sender immediately. Within the bounds of law, the UK based entities in
the Trading arm of the BP Group retain all emails and IMs and monitor
them to ensure compliance with their internal policies and for other
legitimate business purposes.


-----Original Message-----
From: tycho-user-***@eclipse.org
[mailto:tycho-user-***@eclipse.org] On Behalf Of Sievers, Jan
Sent: 19 February 2013 12:56
To: Tycho user list
Subject: Re: [tycho-user] [p2-dev] Tycho 0.16 + Jenkins + NTLM2 proxy

hm. instead of the maven3 jenkins integration, can you use a jenkins
freestyle job and start mvn in a console build step instead ?
just to make sure you do exactly the same thing as on the console where
it works.

Jan

-----Original Message-----
From: tycho-user-***@eclipse.org
[mailto:tycho-user-***@eclipse.org] On Behalf Of Girard-reydet,
Vincent
Sent: Dienstag, 19. Februar 2013 12:31
To: Tycho user list; P2 developer discussions
Subject: Re: [tycho-user] [p2-dev] Tycho 0.16 + Jenkins + NTLM2 proxy

Hi Jan,

Just to clarify things:
- everything happens on my machine (no other machine involved), which is
running Windows 7 (see full env spec in my original email)
- each time the same Maven settings file is used
- Using the same HTTP proxy configuration (I'm using the "native"
settings which are correctly mapped to the proxy to use) I am able to
download and install artifacts in Eclipse. In fact I created the target
file used by Tycho in my Eclipse IDE, it had no problem downloading the
update site contents
- Using the very same Maven settings WORKS OK when I build from my local
console, and FAILS when building inside a Jenkins instance running on my
machine and using my local settings

I have attached to this email two anonymized log files as you requested
in your other answer. The first one shows how it happens on the console,
when it's successful. The other is the Jenkins log file, where I tried
to define
-Dorg.eclipse.ecf.provider.filetransfer.excludeContributors=org.eclipse.
ecf.provider.filetransfer.httpclient as per your suggestion.
Both logs have been taken with maven -e -X.

Note that ${HOME}\.m2\settings.xml does not exist, I'm using only the
global config file at ${MAVEN_HOME}\settings.xml.

Vincent

Vincent Girard-Reydet - LNG Optimisation - BP Gas Marketing Limited, 20
Canada Square, London, E14 5NJ
t: +44 20 7948 5122
BP Gas Marketing Limited, a company registered in England and Wales with
the company number 908982, VAT number GB 365 6789 95 and whose
registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16
7BP This message may contain information that is privileged or
confidential.
If you are not the intended recipient please delete it and inform the
sender immediately. Within the bounds of law, the UK based entities in
the Trading arm of the BP Group retain all emails and IMs and monitor
them to ensure compliance with their internal policies and for other
legitimate business purposes.


-----Original Message-----
From: tycho-user-***@eclipse.org
[mailto:tycho-user-***@eclipse.org] On Behalf Of Sievers, Jan
Sent: 19 February 2013 08:45
To: P2 developer discussions; Tycho user list
Subject: Re: [tycho-user] [p2-dev] Tycho 0.16 + Jenkins + NTLM2 proxy

thanks Scott I think this will help to dig a little deeper.
I don't know how tycho handles the download...so I can't say exactly
what's happening here.
We delegate this to p2/ECF so it should work as if you would
install/update with your IDE.
All we do is configure the HTTP proxy via org.eclipse.core.net (and
register the default java.net.Authenticator, same as
org.eclipse.core.net.ui does IIRC).

two follow-up questions:

1. is there any log/trace so we can check the fallback to the JRE proxy
provider is actually happening in this case?
2. if the JRE proxy fallback happens, any chance different JDKs
(OS/version) may have differing/no support for NTLM proxy
authentication?

@Vincent another thing you can check to help us pin down the issue:
using the same HTTP proxy configurations in your IDE (in Window>Prefs),
can you successfully update/install any bundles e.g. from eclipse.org ?
just to make sure I got this right: you say the build works fine on one
machine but doesn't on another?
what's the difference between those 2 machines (OS, JDK) ?

If the httpclient v4 support is available but not consumed yet,
ultimately we may decide to bundle httpclient v4 with tycho if that
solves the issue.

Regards,
Jan


-----Original Message-----
From: p2-dev-***@eclipse.org [mailto:p2-dev-***@eclipse.org] On
Behalf Of Scott Lewis
Sent: Montag, 18. Februar 2013 23:50
To: p2-***@eclipse.org
Subject: Re: [p2-dev] Tycho 0.16 + Jenkins + NTLM2 proxy

FYI,
- cross-posting to p2-dev -
tycho takes the maven HTTP proxy configuration from settings.xml and
configures p2 accordingly (via proxyservice API in
org.eclipse.core.net) if resolution and download of maven plugins
works but p2 resolution does not, this is a hint that either
a) tycho fails to transfer the proxy info to p2 or
b) p2 / org.eclipse.core.net does not support NTLM proxy
authentication
since nobody reported a) yet I doubt this and suspect b)
I don't know how tycho handles the download...so I can't say exactly
what's happening here.

WRT b...p2 (via ECF filetransfer) does indeed support NTLMv2. There's a
long history to this...see [1]...but the upshot is that ECF filetransfer
has a workaround to not use the apache httpclient 3.1 provider, in the
event of NTLMv2 (since apache httpclient < 4.0 did not support NTLMv2,
and that's what ECF used until recently).
@p2-devs any hints as to b) above?
https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449#c2 may be
related.
@Vincent just in case using apache httpclient v4 would help, this is
not up to you; it's a change which would have to be done in p2 (or at
least in tycho)
No...it's a change that's underway...intended for Kepler: i.e. see this
enhancement [2]. The desire here is that apache httpclient 4.x
reportedly has support for ntlmv2 proxies, where httpclient 3.1 did
not. We have a completed provider based upon httpclient 4.x, and would

appreciate some help with testing against various proxy configurations
in coming integration builds.

But again...since I don't know how/what tycho does WRT downloading, this
might not address your proxy issue.

Thanks,

Scott

[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=252002
[2] https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449
Another thing you should try is to enable advanced HTTP tracing as per
[1] and post the traces
Regards,
Jan
[1] http://wiki.eclipse.org/Equinox_p2_Reporting_Problems
-----Original Message-----
Vincent
Sent: Montag, 18. Februar 2013 18:50
To: Tycho user list
Subject: Re: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy
Hi Jan,
Thank you for your answer. No I'm not using encrypted passwords for
now. I have triple-checked the settings and it's using the proxy (I've
tried removing the section completely and the Jenkins fails to download
the Maven plugins). Here is the proxy section I'm using (passwords,
<proxies>
<proxy>
<id>bp</id>
<active>true</active>
<protocol>http</protocol>
<username>MYUSERNAME</username>
<password>MYPASSWORD</password>
<host>my.proxy.host</host>
<port>808080</port>
<!--<nonProxyHosts>local.net|some.host.com</nonProxyHosts>-->
</proxy>
</proxies>
Note that I have dropped the wagon-http-lightweight-2.2.jar in the
${MAVEN_HOME}/lib/ext folder of my Maven installation as well as in the
folder of Jenkins, just to be sure.
But what I really don't understand is why it works in command line and
not in Jenkins. I don't see where the difference is, as I'm using the
same version of Maven and the same config file. I don't have enough
knowledge of the platform resolution process to understand: where does
Maven stop, where does Tycho stop and where does pure P2 code start in
the platform resolution process, and how does the chain of transmission
of credentials work? I thought that TYCHO-279 had solved the problem,
but from the fix I was not able to completely understand the process.
To me it seems that Tycho should configure the proxy/credentials
resolution in p2 before invoking it to resolve the platform, so the bug
would be in the Tycho/Jenkins interaction, but I may be wrong.
To try the httpclient4, I have to consume the newest version of
org.eclipse.ecf.filetransfer.feature, that's right? What would be the
recommended way of doing (or is it what I have to ask to
Vincent
Vincent Girard-Reydet - LNG Optimisation - BP Gas Marketing Limited,
20 Canada Square, London, E14 5NJ
t: +44 20 7948 5122
BP Gas Marketing Limited, a company registered in England and Wales
with the company number 908982, VAT number GB 365 6789 95 and whose
registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16
7BP This message may contain information that is privileged or
confidential. If you are not the intended recipient please delete it and
inform the sender immediately. Within the bounds of law, the UK based
entities in the Trading arm of the BP Group retain all emails and IMs
and monitor them to ensure compliance with their internal policies and
for other legitimate business purposes.
-----Original Message-----
Sent: 18 February 2013 16:28
To: Tycho user list
Subject: Re: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy
are you using maven-encrypted passwords?
https://bugs.eclipse.org/bugs/show_bug.cgi?id=394023
are you sure you added proxy credentials in settings.xml [1] ?
other than that you may have hit an NTLM limitation in the outdated
apache httpclient 3.x still used by p2.
here is the bug to upgrade to httpclient v4 which seems to indicate
https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449#c2
Regards
Jan
[1] http://maven.apache.org/settings.html#Proxies
Vincent
Sent: Montag, 18. Februar 2013 16:07
Subject: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy
Hi folks,
I'm struggling to configure a Maven build using Tycho on Jenkins
behind my company's proxy. I have a set of modules that build perfectly
well from the console (I just rebuilt everything using a fresh m2 repo
and the build is completely successful). Now I' trying to automate it
with Jenkins (see the full env spec at the end of the email). I have
configured a Maven build in Jenkins. For the local and global settings,
I'm pointing at the exact same settings file I'm using in the console.
This settings file contains the proxy definition, with the username,
password and everything (and working fine in console). The build is
using a private Maven repo.
For my build, I'm using a target platform file (NOT using p2
repositories in the pom.xml), as per the below. This platform is
resolved OK when building in console.
The Jenkins build uses correctly the proxy information to download all
Maven plugins (including Tycho plugins). However, when Tycho starts
resolving the platform, it somehow fails to collect the proxy
information (which it should since TYCHO-279 is now closed). I tried all
the workarounds described to circumvene TYCHO-279 (using the ECF proxy
properties), I tried also using a Nexus repo, nothing worked. I'm a bit
screwed new, it seems to be somehow Jenkins-specific but I don't' have a
clue what's going on.
Would someone be able to help me investigate the problem?
- Windows 7
- NTLM2 proxy
- Maven 3.0.4
- JDK 1.6.39 x86_64
- Jenkins v1.501 (out of the box install, running as a Windows
service)
- Tycho v0.16
<plugin>
<groupId>org.eclipse.tycho</groupId>
<artifactId>target-platform-configuration</artifactId>
<version>${tycho.version}</version>
<configuration>
<target>
<artifact>
<groupId>FAKE_GROUP_ID</groupId>
<artifactId>target</artifactId>
<version>1.0.0</version>
<classifier>indigo</classifier>
</artifact>
</target>
<environments>
<environment>
<os>win32</os>
<ws>win32</ws>
<arch>x86</arch>
</environment>
<environment>
<os>win32</os>
<ws>win32</ws>
<arch>x86_64</arch>
</environment>
</environments>
</configuration>
</plugin>
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?pde
version="3.6"?>
<target name="indigo" sequenceNumber="14"> <locations> <location
includeAllPlatforms="false" includeMode="slicer" includeSource="false"
type="InstallableUnit"> <unit
id="org.eclipse.equinox.server.servletbridge.feature.group"
version="1.0.100.v20110502-42DF9oB5865K5D"/>
<repository
location="http://download.eclipse.org/eclipse/updates/3.7"/>
</location>
<location includeAllPlatforms="false" includeMode="slicer"
includeSource="false" type="InstallableUnit"> <unit
id="org.eclipse.ajdt.feature.group"
version="2.2.0.e37x-RELEASE-20120704-0900"/>
<unit id="org.eclipse.equinox.weaving.sdk.feature.group"
version="1.0.200.I20120427-0800"/>
<repository
location="http://download.eclipse.org/tools/ajdt/37/update"/>
</location>
<location includeAllPlatforms="false" includeMode="slicer"
includeSource="false" type="InstallableUnit"> <unit
id="org.eclipse.sdk.ide" version="3.7.2.M20120208-0800"/> <unit
id="org.eclipse.emf.ecore.feature.group"
version="2.7.1.v20120127-1122"/> <unit
id="org.eclipse.emf.sdk.feature.group"
version="2.7.2.v20120130-0943"/> <unit
id="org.eclipse.emf.compare.feature.group"
version="1.2.2.v20120214-0915"/> <unit
id="org.eclipse.emf.query.feature.group"
version="1.5.0.v20110502-1600-218Z7w311A142A1142"/>
<unit id="org.eclipse.draw2d.feature.group"
version="3.7.2.v20110927-2020-4617w3122212803131"/>
<unit id="org.eclipse.xsd.sdk.feature.group"
version="2.7.1.v20120130-0943"/> <unit
id="org.eclipse.objectteams.otequinox.feature.group"
version="2.0.2.201202051448"/> <unit
id="org.eclipse.equinox.sdk.feature.group"
version="3.7.1.R37x_v20110907-7M7W8i8eNV4WsRkue-4Vq4J6pCyW"/>
<unit id="javax.transaction" version="1.1.1.v201105210645"/>
<unit id="org.eclipse.equinox.executable.feature.group"
version="3.5.0.v20110530-7P7NFUFFLWUl76mart"/>
<unit id="org.eclipse.jdt.feature.group"
version="3.7.1.r371_v20110810-0800-7z8gFcoFMLfTabvKsR5Qm9rBGEBK"/>
<unit id="org.eclipse.rcp.sdk.id" version="3.7.1.M20110909-1335"/>
<repository location="http://download.eclipse.org/releases/indigo"/>
</location>
</locations>
<environment>
<arch>x86</arch>
</environment>
</target>
http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-pl
ugin/1.3/aspectj-maven-plugin-1.3.jar (37 KB at 63.6 KB/sec) [INFO]
C:\jenkins_workspace\test\<MODULE_ID_HIDDEN>\pom.xml
18-Feb-2013 12:30:08
org.apache.commons.httpclient.auth.AuthChallengeProcessor
selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 12:30:08 org.apache.commons.httpclient.HttpMethodDirector
processProxyAuthChallenge
java.lang.RuntimeException: Failed to resolve target definition
C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
cause : Failed to resolve target definition
C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
java.lang.RuntimeException: Failed to resolve target definition
C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
at
org.apache.maven.DefaultMaven.execute(DefaultMaven.java:168)
at
org.jvnet.hudson.maven3.launcher.Maven3Launcher.main(Maven3Launcher.java
:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launche
r.java:329)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:2
39)
at
org.jvnet.hudson.maven3.agent.Maven3Main.launch(Maven3Main.java:158)
at hudson.maven.Maven3Builder.call(Maven3Builder.java:100)
at hudson.maven.Maven3Builder.call(Maven3Builder.java:66)
at hudson.remoting.UserRequest.perform(UserRequest.java:118)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:326)
at
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorS
ervice.java:72)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecuto
r.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
va:918)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.RuntimeException: Failed to resolve target
definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
at
org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileCont
entToTargetPlatform(P2TargetPlatformResolver.java:338)
at
org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.computeTargetPlat
form(P2TargetPlatformResolver.java:207)
at
org.eclipse.tycho.core.resolver.DefaultTychoDependencyResolver.resolvePr
oject(DefaultTychoDependencyResolver.java:94)
at
org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.afterProject
sRead(TychoMavenLifecycleParticipant.java:82)
at
org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:274)
at
org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
... 19 more
org.eclipse.tycho.p2.target.facade.TargetDefinitionResolutionException:
Failed to load metadata repository from location
http://download.eclipse.org/eclipse/updates/3.7
at
org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(Targ
etDefinitionResolver.java:190)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolver.resolveContent(Targ
etDefinitionResolver.java:109)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolverService.resolveFromA
rguments(TargetDefinitionResolverService.java:64)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolverService.getTargetDef
initionContent(TargetDefinitionResolverService.java:54)
at
org.eclipse.tycho.p2.target.TargetPlatformBuilderImpl.addTargetDefinitio
n(TargetPlatformBuilderImpl.java:289)
at
org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileCont
entToTargetPlatform(P2TargetPlatformResolver.java:331)
... 24 more
Caused by: org.eclipse.equinox.p2.core.ProvisionException: HTTP Proxy
Authentication Required:
http://download.eclipse.org/eclipse/updates/3.7/content.xml
at
org.eclipse.equinox.internal.p2.repository.CacheManager.createCache(Cach
eManager.java:192)
at
org.eclipse.tycho.p2.remote.TychoP2RepositoryCacheManager.createCache(Ty
choP2RepositoryCacheManager.java:62)
at
org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataReposi
toryFactory.getLocalFile(SimpleMetadataRepositoryFactory.java:66)
at
org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataReposi
toryFactory.load(SimpleMetadataRepositoryFactory.java:88)
at
org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryMa
nager.factoryLoad(MetadataRepositoryManager.java:57)
at
org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryMan
ager.loadRepository(AbstractRepositoryManager.java:758)
at
org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryMan
ager.loadRepository(AbstractRepositoryManager.java:651)
at
org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryMa
nager.loadRepository(MetadataRepositoryManager.java:96)
at
org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryMa
nager.loadRepository(MetadataRepositoryManager.java:92)
at
org.eclipse.tycho.p2.remote.RemoteMetadataRepositoryManager.loadReposito
ry(RemoteMetadataRepositoryManager.java:82)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(Targ
etDefinitionResolver.java:188)
... 29 more
Proxy auth required
at
org.eclipse.ecf.provider.filetransfer.browse.URLFileSystemBrowser.runReq
uest(URLFileSystemBrowser.java:129)
at
org.eclipse.ecf.provider.filetransfer.browse.AbstractFileSystemBrowser$D
irectoryJob.run(AbstractFileSystemBrowser.java:69)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)
channel stopped
http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plug
in/1.3/aspectj-maven-plugin-1.3.jar (37 KB at 63.6 KB/sec) [INFO]
Scanning for projects...
C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
18-Feb-2013 14:58:28
org.apache.commons.httpclient.auth.AuthChallengeProcessor
selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 14:58:28 org.apache.commons.httpclient.HttpMethodDirector
processProxyAuthChallenge
[WARNING] De-selecting bundles in a target definition file is not
supported. See
http://wiki.eclipse.org/Tycho_Messages_Explained#Target_File_Include_Bun
dles for alternatives.
C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
[WARNING] The following locally built units have been used to resolve
<list of dependencies to local modules> [INFO] Resolving class path of
C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
[WARNING] Dependency from C:\git\ok\<MODULE_ID_HIDDEN> to nested
classpath entry C:\git\ok\<OTHER_MODULE>\icons can not be represented
in Maven model and will not be visible to non-OSGi aware Maven plugins
Thank,
Vincent
Vincent Girard-Reydet - BP Gas Marketing Limited, 20 Canada Square,
London, E14 5NJ BP Gas Marketing Limited, a company registered in
England and Wales with the company number 908982, VAT number GB 365 6789
95 and whose registered office is Chertsey Road, Sunbury on Thames,
Middlesex, TW16 7BP This message may contain information that is
privileged or confidential. If you are not the intended recipient please
delete it and inform the sender immediately. Within the bounds of law,
the UK based entities in the Trading arm of the BP Group retain all
emails and IMs and monitor them to ensure compliance with their internal
policies and for other legitimate business purposes.
_______________________________________________
tycho-user mailing list
https://dev.eclipse.org/mailman/listinfo/tycho-user
_______________________________________________
tycho-user mailing list
https://dev.eclipse.org/mailman/listinfo/tycho-user
_______________________________________________
p2-dev mailing list
https://dev.eclipse.org/mailman/listinfo/p2-dev
_______________________________________________
p2-dev mailing list
p2-***@eclipse.org
https://dev.eclipse.org/mailman/listinfo/p2-dev
Mikhail Kalkov
2013-02-19 15:00:35 UTC
Permalink
Hi Vincent,

If you authenticate to proxy with NTLM and Jenkins runs as a Windows service, it might supply different credentials then your user accont. Besides that, comparing the logs I see that the successful one contains the following line near the top with a path, which is missing in the second log.
[DEBUG] Reading user settings ...

If user settings are indeed empty, I'd suggest to try running Jenkins from your own user account and see if it resolves the issue.

Regards
Mikhail

----- Original Message -----
From: "Vincent Girard-reydet" <Vincent.Girard-***@uk.bp.com>
To: "Tycho user list" <tycho-***@eclipse.org>
Sent: Tuesday, February 19, 2013 3:18:17 PM
Subject: Re: [tycho-user] [p2-dev] Tycho 0.16 + Jenkins + NTLM2 proxy

Hi Jan,

Good call :-) I tried as you suggested (even going to the exact same
folder that works using the console build) and it didn't work either. I
think this must be environmental e.g. the proxy not allowing Jenkins to
authenticate to prevent attacks, or some weirdness like that. That
probably eliminates a pure Tycho or p2 issue.

Vincent

Vincent Girard-Reydet - LNG Optimisation - BP Gas Marketing Limited, 20
Canada Square, London, E14 5NJ
t: +44 20 7948 5122
BP Gas Marketing Limited, a company registered in England and Wales with
the company number 908982, VAT number GB 365 6789 95 and whose
registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16
7BP
This message may contain information that is privileged or confidential.
If you are not the intended recipient please delete it and inform the
sender immediately. Within the bounds of law, the UK based entities in
the Trading arm of the BP Group retain all emails and IMs and monitor
them to ensure compliance with their internal policies and for other
legitimate business purposes.


-----Original Message-----
From: tycho-user-***@eclipse.org
[mailto:tycho-user-***@eclipse.org] On Behalf Of Sievers, Jan
Sent: 19 February 2013 12:56
To: Tycho user list
Subject: Re: [tycho-user] [p2-dev] Tycho 0.16 + Jenkins + NTLM2 proxy

hm. instead of the maven3 jenkins integration, can you use a jenkins
freestyle job and start mvn in a console build step instead ?
just to make sure you do exactly the same thing as on the console where
it works.

Jan

-----Original Message-----
From: tycho-user-***@eclipse.org
[mailto:tycho-user-***@eclipse.org] On Behalf Of Girard-reydet,
Vincent
Sent: Dienstag, 19. Februar 2013 12:31
To: Tycho user list; P2 developer discussions
Subject: Re: [tycho-user] [p2-dev] Tycho 0.16 + Jenkins + NTLM2 proxy

Hi Jan,

Just to clarify things:
- everything happens on my machine (no other machine involved), which is
running Windows 7 (see full env spec in my original email)
- each time the same Maven settings file is used
- Using the same HTTP proxy configuration (I'm using the "native"
settings which are correctly mapped to the proxy to use) I am able to
download and install artifacts in Eclipse. In fact I created the target
file used by Tycho in my Eclipse IDE, it had no problem downloading the
update site contents
- Using the very same Maven settings WORKS OK when I build from my local
console, and FAILS when building inside a Jenkins instance running on my
machine and using my local settings

I have attached to this email two anonymized log files as you requested
in your other answer. The first one shows how it happens on the console,
when it's successful. The other is the Jenkins log file, where I tried
to define
-Dorg.eclipse.ecf.provider.filetransfer.excludeContributors=org.eclipse.
ecf.provider.filetransfer.httpclient as per your suggestion.
Both logs have been taken with maven -e -X.

Note that ${HOME}\.m2\settings.xml does not exist, I'm using only the
global config file at ${MAVEN_HOME}\settings.xml.

Vincent

Vincent Girard-Reydet - LNG Optimisation - BP Gas Marketing Limited, 20
Canada Square, London, E14 5NJ
t: +44 20 7948 5122
BP Gas Marketing Limited, a company registered in England and Wales with
the company number 908982, VAT number GB 365 6789 95 and whose
registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16
7BP This message may contain information that is privileged or
confidential.
If you are not the intended recipient please delete it and inform the
sender immediately. Within the bounds of law, the UK based entities in
the Trading arm of the BP Group retain all emails and IMs and monitor
them to ensure compliance with their internal policies and for other
legitimate business purposes.


-----Original Message-----
From: tycho-user-***@eclipse.org
[mailto:tycho-user-***@eclipse.org] On Behalf Of Sievers, Jan
Sent: 19 February 2013 08:45
To: P2 developer discussions; Tycho user list
Subject: Re: [tycho-user] [p2-dev] Tycho 0.16 + Jenkins + NTLM2 proxy

thanks Scott I think this will help to dig a little deeper.
I don't know how tycho handles the download...so I can't say exactly
what's happening here.
We delegate this to p2/ECF so it should work as if you would
install/update with your IDE.
All we do is configure the HTTP proxy via org.eclipse.core.net (and
register the default java.net.Authenticator, same as
org.eclipse.core.net.ui does IIRC).

two follow-up questions:

1. is there any log/trace so we can check the fallback to the JRE proxy
provider is actually happening in this case?
2. if the JRE proxy fallback happens, any chance different JDKs
(OS/version) may have differing/no support for NTLM proxy
authentication?

@Vincent another thing you can check to help us pin down the issue:
using the same HTTP proxy configurations in your IDE (in Window>Prefs),
can you successfully update/install any bundles e.g. from eclipse.org ?
just to make sure I got this right: you say the build works fine on one
machine but doesn't on another?
what's the difference between those 2 machines (OS, JDK) ?

If the httpclient v4 support is available but not consumed yet,
ultimately we may decide to bundle httpclient v4 with tycho if that
solves the issue.

Regards,
Jan


-----Original Message-----
From: p2-dev-***@eclipse.org [mailto:p2-dev-***@eclipse.org] On
Behalf Of Scott Lewis
Sent: Montag, 18. Februar 2013 23:50
To: p2-***@eclipse.org
Subject: Re: [p2-dev] Tycho 0.16 + Jenkins + NTLM2 proxy

FYI,
- cross-posting to p2-dev -
tycho takes the maven HTTP proxy configuration from settings.xml and
configures p2 accordingly (via proxyservice API in
org.eclipse.core.net) if resolution and download of maven plugins
works but p2 resolution does not, this is a hint that either
a) tycho fails to transfer the proxy info to p2 or
b) p2 / org.eclipse.core.net does not support NTLM proxy
authentication
since nobody reported a) yet I doubt this and suspect b)
I don't know how tycho handles the download...so I can't say exactly
what's happening here.

WRT b...p2 (via ECF filetransfer) does indeed support NTLMv2. There's a
long history to this...see [1]...but the upshot is that ECF filetransfer
has a workaround to not use the apache httpclient 3.1 provider, in the
event of NTLMv2 (since apache httpclient < 4.0 did not support NTLMv2,
and that's what ECF used until recently).
@p2-devs any hints as to b) above?
https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449#c2 may be
related.
@Vincent just in case using apache httpclient v4 would help, this is
not up to you; it's a change which would have to be done in p2 (or at
least in tycho)
No...it's a change that's underway...intended for Kepler: i.e. see this
enhancement [2]. The desire here is that apache httpclient 4.x
reportedly has support for ntlmv2 proxies, where httpclient 3.1 did
not. We have a completed provider based upon httpclient 4.x, and would

appreciate some help with testing against various proxy configurations
in coming integration builds.

But again...since I don't know how/what tycho does WRT downloading, this
might not address your proxy issue.

Thanks,

Scott

[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=252002
[2] https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449
Another thing you should try is to enable advanced HTTP tracing as per
[1] and post the traces
Regards,
Jan
[1] http://wiki.eclipse.org/Equinox_p2_Reporting_Problems
-----Original Message-----
Vincent
Sent: Montag, 18. Februar 2013 18:50
To: Tycho user list
Subject: Re: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy
Hi Jan,
Thank you for your answer. No I'm not using encrypted passwords for
now. I have triple-checked the settings and it's using the proxy (I've
tried removing the section completely and the Jenkins fails to download
the Maven plugins). Here is the proxy section I'm using (passwords,
<proxies>
<proxy>
<id>bp</id>
<active>true</active>
<protocol>http</protocol>
<username>MYUSERNAME</username>
<password>MYPASSWORD</password>
<host>my.proxy.host</host>
<port>808080</port>
<!--<nonProxyHosts>local.net|some.host.com</nonProxyHosts>-->
</proxy>
</proxies>
Note that I have dropped the wagon-http-lightweight-2.2.jar in the
${MAVEN_HOME}/lib/ext folder of my Maven installation as well as in the
folder of Jenkins, just to be sure.
But what I really don't understand is why it works in command line and
not in Jenkins. I don't see where the difference is, as I'm using the
same version of Maven and the same config file. I don't have enough
knowledge of the platform resolution process to understand: where does
Maven stop, where does Tycho stop and where does pure P2 code start in
the platform resolution process, and how does the chain of transmission
of credentials work? I thought that TYCHO-279 had solved the problem,
but from the fix I was not able to completely understand the process.
To me it seems that Tycho should configure the proxy/credentials
resolution in p2 before invoking it to resolve the platform, so the bug
would be in the Tycho/Jenkins interaction, but I may be wrong.
To try the httpclient4, I have to consume the newest version of
org.eclipse.ecf.filetransfer.feature, that's right? What would be the
recommended way of doing (or is it what I have to ask to
Vincent
Vincent Girard-Reydet - LNG Optimisation - BP Gas Marketing Limited,
20 Canada Square, London, E14 5NJ
t: +44 20 7948 5122
BP Gas Marketing Limited, a company registered in England and Wales
with the company number 908982, VAT number GB 365 6789 95 and whose
registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16
7BP This message may contain information that is privileged or
confidential. If you are not the intended recipient please delete it and
inform the sender immediately. Within the bounds of law, the UK based
entities in the Trading arm of the BP Group retain all emails and IMs
and monitor them to ensure compliance with their internal policies and
for other legitimate business purposes.
-----Original Message-----
Sent: 18 February 2013 16:28
To: Tycho user list
Subject: Re: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy
are you using maven-encrypted passwords?
https://bugs.eclipse.org/bugs/show_bug.cgi?id=394023
are you sure you added proxy credentials in settings.xml [1] ?
other than that you may have hit an NTLM limitation in the outdated
apache httpclient 3.x still used by p2.
here is the bug to upgrade to httpclient v4 which seems to indicate
https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449#c2
Regards
Jan
[1] http://maven.apache.org/settings.html#Proxies
Vincent
Sent: Montag, 18. Februar 2013 16:07
Subject: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy
Hi folks,
I'm struggling to configure a Maven build using Tycho on Jenkins
behind my company's proxy. I have a set of modules that build perfectly
well from the console (I just rebuilt everything using a fresh m2 repo
and the build is completely successful). Now I' trying to automate it
with Jenkins (see the full env spec at the end of the email). I have
configured a Maven build in Jenkins. For the local and global settings,
I'm pointing at the exact same settings file I'm using in the console.
This settings file contains the proxy definition, with the username,
password and everything (and working fine in console). The build is
using a private Maven repo.
For my build, I'm using a target platform file (NOT using p2
repositories in the pom.xml), as per the below. This platform is
resolved OK when building in console.
The Jenkins build uses correctly the proxy information to download all
Maven plugins (including Tycho plugins). However, when Tycho starts
resolving the platform, it somehow fails to collect the proxy
information (which it should since TYCHO-279 is now closed). I tried all
the workarounds described to circumvene TYCHO-279 (using the ECF proxy
properties), I tried also using a Nexus repo, nothing worked. I'm a bit
screwed new, it seems to be somehow Jenkins-specific but I don't' have a
clue what's going on.
Would someone be able to help me investigate the problem?
- Windows 7
- NTLM2 proxy
- Maven 3.0.4
- JDK 1.6.39 x86_64
- Jenkins v1.501 (out of the box install, running as a Windows
service)
- Tycho v0.16
<plugin>
<groupId>org.eclipse.tycho</groupId>
<artifactId>target-platform-configuration</artifactId>
<version>${tycho.version}</version>
<configuration>
<target>
<artifact>
<groupId>FAKE_GROUP_ID</groupId>
<artifactId>target</artifactId>
<version>1.0.0</version>
<classifier>indigo</classifier>
</artifact>
</target>
<environments>
<environment>
<os>win32</os>
<ws>win32</ws>
<arch>x86</arch>
</environment>
<environment>
<os>win32</os>
<ws>win32</ws>
<arch>x86_64</arch>
</environment>
</environments>
</configuration>
</plugin>
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?pde
version="3.6"?>
<target name="indigo" sequenceNumber="14"> <locations> <location
includeAllPlatforms="false" includeMode="slicer" includeSource="false"
type="InstallableUnit"> <unit
id="org.eclipse.equinox.server.servletbridge.feature.group"
version="1.0.100.v20110502-42DF9oB5865K5D"/>
<repository
location="http://download.eclipse.org/eclipse/updates/3.7"/>
</location>
<location includeAllPlatforms="false" includeMode="slicer"
includeSource="false" type="InstallableUnit"> <unit
id="org.eclipse.ajdt.feature.group"
version="2.2.0.e37x-RELEASE-20120704-0900"/>
<unit id="org.eclipse.equinox.weaving.sdk.feature.group"
version="1.0.200.I20120427-0800"/>
<repository
location="http://download.eclipse.org/tools/ajdt/37/update"/>
</location>
<location includeAllPlatforms="false" includeMode="slicer"
includeSource="false" type="InstallableUnit"> <unit
id="org.eclipse.sdk.ide" version="3.7.2.M20120208-0800"/> <unit
id="org.eclipse.emf.ecore.feature.group"
version="2.7.1.v20120127-1122"/> <unit
id="org.eclipse.emf.sdk.feature.group"
version="2.7.2.v20120130-0943"/> <unit
id="org.eclipse.emf.compare.feature.group"
version="1.2.2.v20120214-0915"/> <unit
id="org.eclipse.emf.query.feature.group"
version="1.5.0.v20110502-1600-218Z7w311A142A1142"/>
<unit id="org.eclipse.draw2d.feature.group"
version="3.7.2.v20110927-2020-4617w3122212803131"/>
<unit id="org.eclipse.xsd.sdk.feature.group"
version="2.7.1.v20120130-0943"/> <unit
id="org.eclipse.objectteams.otequinox.feature.group"
version="2.0.2.201202051448"/> <unit
id="org.eclipse.equinox.sdk.feature.group"
version="3.7.1.R37x_v20110907-7M7W8i8eNV4WsRkue-4Vq4J6pCyW"/>
<unit id="javax.transaction" version="1.1.1.v201105210645"/>
<unit id="org.eclipse.equinox.executable.feature.group"
version="3.5.0.v20110530-7P7NFUFFLWUl76mart"/>
<unit id="org.eclipse.jdt.feature.group"
version="3.7.1.r371_v20110810-0800-7z8gFcoFMLfTabvKsR5Qm9rBGEBK"/>
<unit id="org.eclipse.rcp.sdk.id" version="3.7.1.M20110909-1335"/>
<repository location="http://download.eclipse.org/releases/indigo"/>
</location>
</locations>
<environment>
<arch>x86</arch>
</environment>
</target>
http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-pl
ugin/1.3/aspectj-maven-plugin-1.3.jar (37 KB at 63.6 KB/sec) [INFO]
C:\jenkins_workspace\test\<MODULE_ID_HIDDEN>\pom.xml
18-Feb-2013 12:30:08
org.apache.commons.httpclient.auth.AuthChallengeProcessor
selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 12:30:08 org.apache.commons.httpclient.HttpMethodDirector
processProxyAuthChallenge
java.lang.RuntimeException: Failed to resolve target definition
C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
cause : Failed to resolve target definition
C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
java.lang.RuntimeException: Failed to resolve target definition
C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
at
org.apache.maven.DefaultMaven.execute(DefaultMaven.java:168)
at
org.jvnet.hudson.maven3.launcher.Maven3Launcher.main(Maven3Launcher.java
:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launche
r.java:329)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:2
39)
at
org.jvnet.hudson.maven3.agent.Maven3Main.launch(Maven3Main.java:158)
at hudson.maven.Maven3Builder.call(Maven3Builder.java:100)
at hudson.maven.Maven3Builder.call(Maven3Builder.java:66)
at hudson.remoting.UserRequest.perform(UserRequest.java:118)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:326)
at
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorS
ervice.java:72)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecuto
r.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
va:918)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.RuntimeException: Failed to resolve target
definition C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
at
org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileCont
entToTargetPlatform(P2TargetPlatformResolver.java:338)
at
org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.computeTargetPlat
form(P2TargetPlatformResolver.java:207)
at
org.eclipse.tycho.core.resolver.DefaultTychoDependencyResolver.resolvePr
oject(DefaultTychoDependencyResolver.java:94)
at
org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.afterProject
sRead(TychoMavenLifecycleParticipant.java:82)
at
org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:274)
at
org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
... 19 more
org.eclipse.tycho.p2.target.facade.TargetDefinitionResolutionException:
Failed to load metadata repository from location
http://download.eclipse.org/eclipse/updates/3.7
at
org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(Targ
etDefinitionResolver.java:190)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolver.resolveContent(Targ
etDefinitionResolver.java:109)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolverService.resolveFromA
rguments(TargetDefinitionResolverService.java:64)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolverService.getTargetDef
initionContent(TargetDefinitionResolverService.java:54)
at
org.eclipse.tycho.p2.target.TargetPlatformBuilderImpl.addTargetDefinitio
n(TargetPlatformBuilderImpl.java:289)
at
org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileCont
entToTargetPlatform(P2TargetPlatformResolver.java:331)
... 24 more
Caused by: org.eclipse.equinox.p2.core.ProvisionException: HTTP Proxy
Authentication Required:
http://download.eclipse.org/eclipse/updates/3.7/content.xml
at
org.eclipse.equinox.internal.p2.repository.CacheManager.createCache(Cach
eManager.java:192)
at
org.eclipse.tycho.p2.remote.TychoP2RepositoryCacheManager.createCache(Ty
choP2RepositoryCacheManager.java:62)
at
org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataReposi
toryFactory.getLocalFile(SimpleMetadataRepositoryFactory.java:66)
at
org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataReposi
toryFactory.load(SimpleMetadataRepositoryFactory.java:88)
at
org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryMa
nager.factoryLoad(MetadataRepositoryManager.java:57)
at
org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryMan
ager.loadRepository(AbstractRepositoryManager.java:758)
at
org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryMan
ager.loadRepository(AbstractRepositoryManager.java:651)
at
org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryMa
nager.loadRepository(MetadataRepositoryManager.java:96)
at
org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryMa
nager.loadRepository(MetadataRepositoryManager.java:92)
at
org.eclipse.tycho.p2.remote.RemoteMetadataRepositoryManager.loadReposito
ry(RemoteMetadataRepositoryManager.java:82)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(Targ
etDefinitionResolver.java:188)
... 29 more
Proxy auth required
at
org.eclipse.ecf.provider.filetransfer.browse.URLFileSystemBrowser.runReq
uest(URLFileSystemBrowser.java:129)
at
org.eclipse.ecf.provider.filetransfer.browse.AbstractFileSystemBrowser$D
irectoryJob.run(AbstractFileSystemBrowser.java:69)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)
channel stopped
http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plug
in/1.3/aspectj-maven-plugin-1.3.jar (37 KB at 63.6 KB/sec) [INFO]
Scanning for projects...
C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
18-Feb-2013 14:58:28
org.apache.commons.httpclient.auth.AuthChallengeProcessor
selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 14:58:28 org.apache.commons.httpclient.HttpMethodDirector
processProxyAuthChallenge
[WARNING] De-selecting bundles in a target definition file is not
supported. See
http://wiki.eclipse.org/Tycho_Messages_Explained#Target_File_Include_Bun
dles for alternatives.
C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
[WARNING] The following locally built units have been used to resolve
<list of dependencies to local modules> [INFO] Resolving class path of
C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
[WARNING] Dependency from C:\git\ok\<MODULE_ID_HIDDEN> to nested
classpath entry C:\git\ok\<OTHER_MODULE>\icons can not be represented
in Maven model and will not be visible to non-OSGi aware Maven plugins
Thank,
Vincent
Vincent Girard-Reydet - BP Gas Marketing Limited, 20 Canada Square,
London, E14 5NJ BP Gas Marketing Limited, a company registered in
England and Wales with the company number 908982, VAT number GB 365 6789
95 and whose registered office is Chertsey Road, Sunbury on Thames,
Middlesex, TW16 7BP This message may contain information that is
privileged or confidential. If you are not the intended recipient please
delete it and inform the sender immediately. Within the bounds of law,
the UK based entities in the Trading arm of the BP Group retain all
emails and IMs and monitor them to ensure compliance with their internal
policies and for other legitimate business purposes.
_______________________________________________
tycho-user mailing list
https://dev.eclipse.org/mailman/listinfo/tycho-user
_______________________________________________
tycho-user mailing list
https://dev.eclipse.org/mailman/listinfo/tycho-user
_______________________________________________
p2-dev mailing list
https://dev.eclipse.org/mailman/listinfo/p2-dev
_______________________________________________
p2-dev mailing list
p2-***@eclipse.org
https://dev.eclipse.org/mailman/listinfo/p2-dev
_______________________________________________
tycho-user mailing list
tycho-***@eclipse.org
https://dev.eclipse.org/mailman/listinfo/tycho-user

Loading...