[tycho-user] Resolve Maven plugin dependencies from target platform?

[Andreas Sewe]

Hi,

AFAIK, Tycho hooks into Maven's dependency resolution and, through some
system-scope dependency magic, makes Maven pick up project dependencies
as declared in the target platform.

Now, I wonder whether it's possible to also resolve a *Maven* plugin's
dependencies from the target platform?

Why would one want to do that? Well, in our build we use the
xtext-maven-plugin which requires a couple of plugin dependencies [1].
All these dependencies are readily available from our target platform.
However, I still have to hunt down the "equivalent" JARs on Maven
central and maintain their versions in the POM. Keeping this in sync is
not fun.

Hence, I am wondering whether Tycho could hook into Maven's dependency
resolution in such a way that plugin dependencies could also come from
the target platform?

Best wishes,

Andreas

[1]
<https://git.eclipse.org/c/recommenders/org.eclipse.recommenders.git/tree/plugins/org.eclipse.recommenders.stacktraces.model/pom.xml?id=v2.1.10#n44>
--
Codetrails GmbH
The knowledge transfer company

Robert-Bosch-Str. 7, 64293 Darmstadt
Phone: [phone]
Mobile: [phone]
http://www.codetrails.com/

Managing Director: Dr. Marcel Bruch
Handelsregister: Darmstadt HRB 91940

[Igor Fedorenko]

This is theoretically possible, but Tycho does not do this right now and
I am not aware of any attempts to implement this either.

I am not even sure you need/want to make this part of Tycho. What you
are looking for, basically, is Maven repository layout implementation
that is able to perform jar and pom (!) artifact lookup based on maven
artifactcoordinates. This is quite different from what Tycho does not
project dependencies.

The difficult part is, of course, figuring out how to represent OSGi
dependencies in Maven, but maybe you can get away by simply ignoring the
problem.

If you really need just couple of dependencies, however, it is probably
easier to manually publish the required artifacts to a Maven repository.
This is how we make equinox and jdt compiler available for Tycho [2].

[2]
http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/tree/tycho-releng/pom.xml

--
Regards,
Igor

Post by Andreas Sewe
Hi,
AFAIK, Tycho hooks into Maven's dependency resolution and, through some
system-scope dependency magic, makes Maven pick up project dependencies
as declared in the target platform.
Now, I wonder whether it's possible to also resolve a *Maven* plugin's
dependencies from the target platform?
Why would one want to do that? Well, in our build we use the
xtext-maven-plugin which requires a couple of plugin dependencies [1].
All these dependencies are readily available from our target platform.
However, I still have to hunt down the "equivalent" JARs on Maven
central and maintain their versions in the POM. Keeping this in sync is
not fun.
Hence, I am wondering whether Tycho could hook into Maven's dependency
resolution in such a way that plugin dependencies could also come from
the target platform?
Best wishes,
Andreas
[1]
<https://git.eclipse.org/c/recommenders/org.eclipse.recommenders.git/tree/plugins/org.eclipse.recommenders.stacktraces.model/pom.xml?id=v2.1.10#n44>

[Andreas Sewe]

Hi Igor,

thank you for your insights.

Post by Igor Fedorenko
This is theoretically possible, but Tycho does not do this right now and
I am not aware of any attempts to implement this either.
I am not even sure you need/want to make this part of Tycho. What you
are looking for, basically, is Maven repository layout implementation
that is able to perform jar and pom (!) artifact lookup based on maven
artifactcoordinates. This is quite different from what Tycho does not
project dependencies.
The difficult part is, of course, figuring out how to represent OSGi
dependencies in Maven, but maybe you can get away by simply ignoring the
problem.
If you really need just couple of dependencies, however, it is probably
easier to manually publish the required artifacts to a Maven repository.
This is how we make equinox and jdt compiler available for Tycho [2].

I think the situation is different in case of the xtext-maven-plugin
(which may be a weird outlier, admittedly).

In the case of the JDT compiler you have an OSGi bundle that you want to
make available to your Maven build. A "mvn deploy:deploy-file" does just
that (of course, where the POM comes from is interesting). The JDT
compiler bundle is needed only as a dependency of the plugin, though.

In the case of the xtext-maven-plugin, however, some bundles like
org.eclipse.emf.ecore.xcore.lib are needed both by the plugin [1] (a
code generator) and by the project that the code is generated for [2].
Ideally, the project's and the plugin's org.eclipse.emf.ecore.xcore.lib
dependency would be identical. At the very least, their versions should
not diverge too much, as then the generated code would no longer work
with the org.eclipse.emf.ecore.xcore.lib used to compile/run it.

However, keeping the versions in sync is cumbersome, as the project
dependencies' versions are governed by the target platform and the
plugin dependencies' versions are governed by the pom.xml. Hope that
this explains a bit better what my perceived problem is (Disclaimer: I
am no Xtext/Xcore expert).

Best wishes,

Andreas

[1]
<https://git.eclipse.org/c/recommenders/org.eclipse.recommenders.git/tree/plugins/org.eclipse.recommenders.stacktraces.model/pom.xml?id=v2.1.10#n44>
[2]
<https://git.eclipse.org/c/recommenders/org.eclipse.recommenders.git/tree/plugins/org.eclipse.recommenders.stacktraces.model/META-INF/MANIFEST.MF?id=v2.1.10#n9>
--
Codetrails GmbH
The knowledge transfer company

Robert-Bosch-Str. 7, 64293 Darmstadt
Phone: [phone]
Mobile: [phone]
http://www.codetrails.com/

Managing Director: Dr. Marcel Bruch
Handelsregister: Darmstadt HRB 91940

[Igor Fedorenko]

Personally, I consider this an antipatern. Such tight coupling between
code generator, runtime and generated code is just asking for troubles.

I don't see how to make this work. I don't believe there way to express
"this plugin depends on the same version of xtext runtime as the project
being built". If this was plain Maven project you could use
${xtext-version} property to link dependency and plugin/dependency
versions, but with Tycho this is not possible because xtext version
isn't known until well into the build.

--
Regards,
Igor

Post by Andreas Sewe
Hi Igor,
thank you for your insights.

Post by Igor Fedorenko
This is theoretically possible, but Tycho does not do this right now and
I am not aware of any attempts to implement this either.
I am not even sure you need/want to make this part of Tycho. What you
are looking for, basically, is Maven repository layout implementation
that is able to perform jar and pom (!) artifact lookup based on maven
artifactcoordinates. This is quite different from what Tycho does not
project dependencies.
The difficult part is, of course, figuring out how to represent OSGi
dependencies in Maven, but maybe you can get away by simply ignoring the
problem.
If you really need just couple of dependencies, however, it is probably
easier to manually publish the required artifacts to a Maven repository.
This is how we make equinox and jdt compiler available for Tycho [2].

I think the situation is different in case of the xtext-maven-plugin
(which may be a weird outlier, admittedly).
In the case of the JDT compiler you have an OSGi bundle that you want to
make available to your Maven build. A "mvn deploy:deploy-file" does just
that (of course, where the POM comes from is interesting). The JDT
compiler bundle is needed only as a dependency of the plugin, though.
In the case of the xtext-maven-plugin, however, some bundles like
org.eclipse.emf.ecore.xcore.lib are needed both by the plugin [1] (a
code generator) and by the project that the code is generated for [2].
Ideally, the project's and the plugin's org.eclipse.emf.ecore.xcore.lib
dependency would be identical. At the very least, their versions should
not diverge too much, as then the generated code would no longer work
with the org.eclipse.emf.ecore.xcore.lib used to compile/run it.
However, keeping the versions in sync is cumbersome, as the project
dependencies' versions are governed by the target platform and the
plugin dependencies' versions are governed by the pom.xml. Hope that
this explains a bit better what my perceived problem is (Disclaimer: I
am no Xtext/Xcore expert).
Best wishes,
Andreas
[1]
<https://git.eclipse.org/c/recommenders/org.eclipse.recommenders.git/tree/plugins/org.eclipse.recommenders.stacktraces.model/pom.xml?id=v2.1.10#n44>
[2]
<https://git.eclipse.org/c/recommenders/org.eclipse.recommenders.git/tree/plugins/org.eclipse.recommenders.stacktraces.model/META-INF/MANIFEST.MF?id=v2.1.10#n9>

[Andreas Sewe]

Hi Igor,

Post by Igor Fedorenko
Personally, I consider this an antipatern. Such tight coupling between
code generator, runtime and generated code is just asking for troubles.

I tend to agree. Based on my limited understanding of Xtext, the issue
arises because at least one of the xtext-maven-plugin's dependencies is
the language definition itself, which is also used/needed at runtime.

But maybe I need to better understand how closely the
code-generation-time language definition and the runtime language
definition need to match first. I'll ask in the Xtext forum. Thanks for
your comments; they will be a good input for my inquiry there.

Post by Igor Fedorenko
I don't see how to make this work. I don't believe there way to express
"this plugin depends on the same version of xtext runtime as the project
being built". If this was plain Maven project you could use
${xtext-version} property to link dependency and plugin/dependency
versions, but with Tycho this is not possible because xtext version
isn't known until well into the build.

Does that mean that Maven resolves plugin versions for all the projects
in the reactor at the very start rather than when building each project.
If that is the case, then I can see why this feature would be very hard,
if not impossible to implement.

Best wishes,

Andreas
--
Codetrails GmbH
The knowledge transfer company

Robert-Bosch-Str. 7, 64293 Darmstadt
Phone: [phone]
Mobile: [phone]
http://www.codetrails.com/

Managing Director: Dr. Marcel Bruch
Handelsregister: Darmstadt HRB 91940

[Sven Efftinge]

Hi Andreas,

there is no inherent coupling between the libs needed by the plugin and the libs needed by the generated code.
It is just that you are generating an Xtext language infrastructure using an Xtext language infrastructure (i.e. the *.xtext grammar language).
So of course the dependencies are very alike, but this is just a result of the bootstrapping. There are clear version constraints for both the plugin and the
generated bundle projects. I hope this is understandable.

I still see that it would be convenient to reuse the dependency definitions but I don’t see how we could do something for that in Xtext.

Cheers,
Sven

Post by Andreas Sewe
Hi Igor,

Post by Igor Fedorenko
Personally, I consider this an antipatern. Such tight coupling between
code generator, runtime and generated code is just asking for troubles.

I tend to agree. Based on my limited understanding of Xtext, the issue
arises because at least one of the xtext-maven-plugin's dependencies is
the language definition itself, which is also used/needed at runtime.
But maybe I need to better understand how closely the
code-generation-time language definition and the runtime language
definition need to match first. I'll ask in the Xtext forum. Thanks for
your comments; they will be a good input for my inquiry there.

Post by Igor Fedorenko
I don't see how to make this work. I don't believe there way to express
"this plugin depends on the same version of xtext runtime as the project
being built". If this was plain Maven project you could use
${xtext-version} property to link dependency and plugin/dependency
versions, but with Tycho this is not possible because xtext version
isn't known until well into the build.

Does that mean that Maven resolves plugin versions for all the projects
in the reactor at the very start rather than when building each project.
If that is the case, then I can see why this feature would be very hard,
if not impossible to implement.
Best wishes,
Andreas
--
Codetrails GmbH
The knowledge transfer company
Robert-Bosch-Str. 7, 64293 Darmstadt
Phone: [phone]
Mobile: [phone]
http://www.codetrails.com/
Managing Director: Dr. Marcel Bruch
Handelsregister: Darmstadt HRB 91940
_______________________________________________
tycho-user mailing list
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/tycho-user

[Andreas Sewe]

Hi Sven,

thank you for your reply.

Post by Sven Efftinge
there is no inherent coupling between the libs needed by the plugin and the libs needed by the generated code.
It is just that you are generating an Xtext language infrastructure using an Xtext language infrastructure (i.e. the *.xtext grammar language).
So of course the dependencies are very alike, but this is just a result of the bootstrapping. There are clear version constraints for both the plugin and the
generated bundle projects. I hope this is understandable.

Maybe you can shed some more light on the way versions are handled.

I have an Xcore model like the following [1]. Using the
xtext-maven-plugin defined in my pom.xml [2] takes this model as input
and generates Java code based on the Xcore language definition found in
the plugin's dependencies. This Java code assumes that the Xcore runtime
library in a certain version (range) is present. Correct?

Now, as far as I know, you can leave it to Xtext to rewrite the
MANIFEST.MF and add the appropriate version ranges:

@GenModel(bundleManifest="true")

Even if this directive would work with the xtext-maven-plugin (it has no
effect in my build. Bug?), letting the code generation step rewrite the
MANIFEST.MF clashes with with the way Tycho works (at least to my
limited understanding of Tycho). By the time the rewrite happens Tycho
has already read the old, non-rewritten MANFIEST.MF and build its mental
model of the project dependencies based on that.

Moreover, whatever the xtext-maven-plugin generates during a "mvn clean
install" normally just won't be checked into SCM.

So, automatically letting the code generator fill in the version ranges
in the MANIFEST.MF doesn't seem to work well -- at least not during the
code-generation step part of a "mvn clean install".

What is the recommended workflow then? Trigger code generation manually
every once in a while and explicitly check in the generated MANIFEST.MF?

Post by Sven Efftinge
I still see that it would be convenient to reuse the dependency definitions but I don’t see how we could do something for that in Xtext.

One way would be let Tycho resolve the xtext-maven-plugin's dependencies
to the bundles defined in the target platform (hence my original
question), but as Igor said this might be hard/impossible.

Best wishes,

Andreas

[1]
<https://git.eclipse.org/c/recommenders/org.eclipse.recommenders.git/tree/plugins/org.eclipse.recommenders.stacktraces.model/models/stacktraces.xcore?id=v2.1.10>
[2]
<https://git.eclipse.org/c/recommenders/org.eclipse.recommenders.git/tree/plugins/org.eclipse.recommenders.stacktraces.model/pom.xml?id=v2.1.10>
--
Codetrails GmbH
The knowledge transfer company

Robert-Bosch-Str. 7, 64293 Darmstadt
Phone: [phone]
Mobile: [phone]
http://www.codetrails.com/

Managing Director: Dr. Marcel Bruch
Handelsregister: Darmstadt HRB 91940

[Sven Efftinge]

Hi Andreas,

it is a ‘feature’ of EMF’s Ecore generator (used by Xcore) to touch the MANIFEST.MF.
I guess it is best to file a bugzilla against EMF (Xcore component) and discuss your use case there.

Sven

Post by Andreas Sewe
Hi Sven,
thank you for your reply.

Post by Sven Efftinge
there is no inherent coupling between the libs needed by the plugin and the libs needed by the generated code.
It is just that you are generating an Xtext language infrastructure using an Xtext language infrastructure (i.e. the *.xtext grammar language).
So of course the dependencies are very alike, but this is just a result of the bootstrapping. There are clear version constraints for both the plugin and the
generated bundle projects. I hope this is understandable.

Maybe you can shed some more light on the way versions are handled.
I have an Xcore model like the following [1]. Using the
xtext-maven-plugin defined in my pom.xml [2] takes this model as input
and generates Java code based on the Xcore language definition found in
the plugin's dependencies. This Java code assumes that the Xcore runtime
library in a certain version (range) is present. Correct?
Now, as far as I know, you can leave it to Xtext to rewrite the
@GenModel(bundleManifest="true")
Even if this directive would work with the xtext-maven-plugin (it has no
effect in my build. Bug?), letting the code generation step rewrite the
MANIFEST.MF clashes with with the way Tycho works (at least to my
limited understanding of Tycho). By the time the rewrite happens Tycho
has already read the old, non-rewritten MANFIEST.MF and build its mental
model of the project dependencies based on that.
Moreover, whatever the xtext-maven-plugin generates during a "mvn clean
install" normally just won't be checked into SCM.
So, automatically letting the code generator fill in the version ranges
in the MANIFEST.MF doesn't seem to work well -- at least not during the
code-generation step part of a "mvn clean install".
What is the recommended workflow then? Trigger code generation manually
every once in a while and explicitly check in the generated MANIFEST.MF?

Post by Sven Efftinge
I still see that it would be convenient to reuse the dependency definitions but I don’t see how we could do something for that in Xtext.

One way would be let Tycho resolve the xtext-maven-plugin's dependencies
to the bundles defined in the target platform (hence my original
question), but as Igor said this might be hard/impossible.
Best wishes,
Andreas
[1]
<https://git.eclipse.org/c/recommenders/org.eclipse.recommenders.git/tree/plugins/org.eclipse.recommenders.stacktraces.model/models/stacktraces.xcore?id=v2.1.10>
[2]
<https://git.eclipse.org/c/recommenders/org.eclipse.recommenders.git/tree/plugins/org.eclipse.recommenders.stacktraces.model/pom.xml?id=v2.1.10>
--
Codetrails GmbH
The knowledge transfer company
Robert-Bosch-Str. 7, 64293 Darmstadt
Phone: [phone]
Mobile: [phone]
http://www.codetrails.com/
Managing Director: Dr. Marcel Bruch
Handelsregister: Darmstadt HRB 91940
_______________________________________________
tycho-user mailing list
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/tycho-user

[Igor Fedorenko]

Post by Andreas Sewe

Post by Igor Fedorenko
I don't see how to make this work. I don't believe there way to
express "this plugin depends on the same version of xtext runtime
as the project being built". If this was plain Maven project you
could use ${xtext-version} property to link dependency and
plugin/dependency versions, but with Tycho this is not possible
because xtext version isn't known until well into the build.

Does that mean that Maven resolves plugin versions for all the
projects in the reactor at the very start rather than when building
each project. If that is the case, then I can see why this feature
would be very hard, if not impossible to implement.

Maven plugins and their dependencies are resolved before the build
starts, iirc, and project dependencies are normally resolved right
before execution of individual plugin goals.

But even if plugins were resolved after project dependencies, how would
you express "this plugin depends on the same version of xtext runtime as
the project being built"? Can you provide pom.xml snippet?

--
Regards,
Igor

[Andreas Sewe]

Hi Igor,

Post by Igor Fedorenko
But even if plugins were resolved after project dependencies, how would
you express "this plugin depends on the same version of xtext runtime as
the project being built"? Can you provide pom.xml snippet?

ah, I don't want to express "this plugin depends on the same version of
xtext runtime as the project being built" but rather "this plugin
depends on the xtext runtime from the target platform configured for
this project".

Something like

<plugin>
...
<dependencies>
<dependency>
<artifactId>org.eclipse.emf.ecore.xcore</artifactId>
<!-- Triggers some resolution magic --<
<scope>system</scope>
<systemPath>target-platform</systemPath>
</dependency>
</dependencies>
</plugin>

I realize that overloading system-scope in this way is ugly, but Tycho
uses the system scope for its special treatment in other places already.
Doing this in a nicer way probably would require some sort of POM 5.0,
though.

Hope that explains what I am after.

Andreas
--
Codetrails GmbH
The knowledge transfer company

Robert-Bosch-Str. 7, 64293 Darmstadt
Phone: [phone]
Mobile: [phone]
http://www.codetrails.com/

Managing Director: Dr. Marcel Bruch
Handelsregister: Darmstadt HRB 91940

[Igor Fedorenko]

Tycho does not use system scope the way to describe. Tycho never
resolves system-scoped dependencies from the build target platform.

--
Regards,
Igor

Post by Andreas Sewe
Hi Igor,

Post by Igor Fedorenko
But even if plugins were resolved after project dependencies, how would
you express "this plugin depends on the same version of xtext runtime as
the project being built"? Can you provide pom.xml snippet?

ah, I don't want to express "this plugin depends on the same version of
xtext runtime as the project being built" but rather "this plugin
depends on the xtext runtime from the target platform configured for
this project".
Something like
<plugin>
...
<dependencies>
<dependency>
<artifactId>org.eclipse.emf.ecore.xcore</artifactId>
<!-- Triggers some resolution magic --<
<scope>system</scope>
<systemPath>target-platform</systemPath>
</dependency>
</dependencies>
</plugin>
I realize that overloading system-scope in this way is ugly, but Tycho
uses the system scope for its special treatment in other places already.
Doing this in a nicer way probably would require some sort of POM 5.0,
though.
Hope that explains what I am after.
Andreas

[Andreas Sewe]

Hi,

Post by Igor Fedorenko
Tycho does not use system scope the way to describe. Tycho never
resolves system-scoped dependencies from the build target platform.

true. But AFAIK it internally represents bundle dependencies as
system-scoped dependencies, does it not? At least that's the way "mvn

Post by Igor Fedorenko
[INFO] org.eclipse.recommenders:org.eclipse.recommenders.utils:eclipse-plugin:2.1.11-SNAPSHOT
[INFO] +- p2.eclipse-plugin:com.google.guava:jar:15.0.0.v201403281430:system
[INFO] +- p2.eclipse-plugin:org.eclipse.osgi:jar:3.10.2.v20141103-1919:system
[INFO] +- p2.eclipse-plugin:org.eclipse.core.contenttype:jar:3.4.200.v20140207-1251:system
[INFO] +- p2.eclipse-plugin:org.eclipse.equinox.preferences:jar:3.5.200.v20140224-1527:system
[INFO] +- p2.eclipse-plugin:org.eclipse.equinox.registry:jar:3.5.400.v20140428-1507:system
[INFO] +- p2.eclipse-plugin:org.eclipse.equinox.common:jar:3.6.200.v20130402-1505:system
[INFO] +- p2.eclipse-plugin:org.eclipse.core.jobs:jar:3.6.1.v20141014-1248:system
[INFO] +- p2.eclipse-plugin:org.eclipse.core.runtime:jar:3.10.0.v20140318-2214:system
[INFO] +- p2.eclipse-plugin:org.eclipse.equinox.app:jar:1.3.200.v20130910-1609:system
[INFO] +- p2.eclipse-plugin:org.apache.commons.io:jar:2.0.1.v201105210651:system
[INFO] +- p2.eclipse-plugin:com.google.gson:jar:2.2.4.v201311231704:system
[INFO] \- p2.eclipse-plugin:org.apache.commons.lang3:jar:3.1.0.v201403281430:system

Best wishes,

Andreas
--
Codetrails GmbH
The knowledge transfer company

Robert-Bosch-Str. 7, 64293 Darmstadt
Phone: [phone]
Mobile: [phone]
http://www.codetrails.com/

Managing Director: Dr. Marcel Bruch
Handelsregister: Darmstadt HRB 91940

[Igor Fedorenko]

In case of bundle projects, dependency information comes from bundle
manifest and resolved dependencies are then reflected to MavenProject model.

In your case, you want dependency information come from MavenProject
model. You may be able to use custom scope or weird packaging type to
fake this, but I am not too sure if current maven versions support that.

--
Regards,
Igor

Post by Andreas Sewe
Hi,

Post by Igor Fedorenko
Tycho does not use system scope the way to describe. Tycho never
resolves system-scoped dependencies from the build target platform.

true. But AFAIK it internally represents bundle dependencies as
system-scoped dependencies, does it not? At least that's the way "mvn

Post by Igor Fedorenko
[INFO] org.eclipse.recommenders:org.eclipse.recommenders.utils:eclipse-plugin:2.1.11-SNAPSHOT
[INFO] +- p2.eclipse-plugin:com.google.guava:jar:15.0.0.v201403281430:system
[INFO] +- p2.eclipse-plugin:org.eclipse.osgi:jar:3.10.2.v20141103-1919:system
[INFO] +- p2.eclipse-plugin:org.eclipse.core.contenttype:jar:3.4.200.v20140207-1251:system
[INFO] +- p2.eclipse-plugin:org.eclipse.equinox.preferences:jar:3.5.200.v20140224-1527:system
[INFO] +- p2.eclipse-plugin:org.eclipse.equinox.registry:jar:3.5.400.v20140428-1507:system
[INFO] +- p2.eclipse-plugin:org.eclipse.equinox.common:jar:3.6.200.v20130402-1505:system
[INFO] +- p2.eclipse-plugin:org.eclipse.core.jobs:jar:3.6.1.v20141014-1248:system
[INFO] +- p2.eclipse-plugin:org.eclipse.core.runtime:jar:3.10.0.v20140318-2214:system
[INFO] +- p2.eclipse-plugin:org.eclipse.equinox.app:jar:1.3.200.v20130910-1609:system
[INFO] +- p2.eclipse-plugin:org.apache.commons.io:jar:2.0.1.v201105210651:system
[INFO] +- p2.eclipse-plugin:com.google.gson:jar:2.2.4.v201311231704:system
[INFO] \- p2.eclipse-plugin:org.apache.commons.lang3:jar:3.1.0.v201403281430:system

Best wishes,
Andreas