Subject: javadocs generated by javahelper include jquery

From: Thomas Koch <[email-masked]>
Date: Mon, 09 Jul 2018 21:48:20 +0200

Package: javahelper
Severity: important
Tags: newcomer

The command jh_build calls javadoc. The output of javadoc contains a copy of
jquery which leads to lintian error:
https://lintian.debian.org/tags/embedded-javascript-library.html

One solution would be if javahelper would strip the embedded jquery library
and replace it with a symlink to the version from the jquery debian package
and add a dependency on the jquery package to the -doc package.

The problem is that we'd need to rebuild all javadoc packages if the jquery
package gets updated in an incompatible way.

Therefor it would be better if javadoc had an option to build the docs without
jquery, but I have not found such an option. What does javadoc need jquery for
in a world with HTML5?

There was bug 883981 about exactly this ("openjdk-9: embeds jquery.js when
creating javadoc") which got autoclosed.

I made this bug important since it leads to policy violation errors in the
generated packages. It is not feasible to fix this bug in all the generated
- -doc packages manually.

Java 11 should at least come with an updated jquery:
https://bugs.openjdk.java.net/browse/JDK-8204865

---

From: Christoph Berg <[email-masked]>
Date: Mon, 9 Jul 2018 22:41:18 +0200

Re: Thomas Koch 2018-07-09 <[email-masked]>
> package gets updated in an incompatible way.

The problem would be even bigger if people did the transformation
manually and then jquery broke. If the symlinks are handled by
javahandler, a binNMU would take of everything, without manual
intervention.

> jquery, but I have not found such an option. What does javadoc need jquery for

Or even better, have javadoc put in the symlink.

Christoph

---

From: Emmanuel Bourg <[email-masked]>
Date: Mon, 9 Jul 2018 23:01:09 +0200

> Or even better, have javadoc put in the symlink.

Not a good idea. The javadoc generated would no longer be usable outside
Debian. Developers would no longer be able to generate the javadoc of
their libraries a upload it to a web server.

Emmanuel Bourg

---

From: Markus Koschany <[email-masked]>
Date: Mon, 9 Jul 2018 23:14:28 +0200

> their libraries a upload it to a web server.

I think this should work in the context of Debian. It's kind of funny
that we always argue about the usage of Debian packages (e.g. the Spring
framework clearly intended to be for web development, affected by
numerous security vulnerabilities but it is assumed nobody uses it in
production environments but only for building unrelated packages), still
we care about the use case of shipping javadoc outside of a Debian
environment.

In my opinion this issue should be fixed at the OpenJDK level. I am the
one who filed bug #883981 and I'm a bit disappointed that it was closed
automatically, not even reassigned to openjdk-10. IMO it would be
possible to create a javahelper/maven-debian-helper/etc option to choose
between embedding jquery into javadoc or symlinking to the system
libraries. I believe the use case of viewing javadoc outside of a Debian
system is negligible and we should just symlink jquery.

Markus

---

From: Emmanuel Bourg <[email-masked]>
Date: Mon, 9 Jul 2018 23:26:16 +0200

Le 09/07/2018 à 23:14, Markus Koschany a écrit :

> I believe the use case of viewing javadoc outside of a Debian

Viewing an API documentation from a lib*-java-doc package outside of a
Debian system is indeed a negligible use case. But if the javadoc tool
was to be patched to link any javadoc generated with the system jquery,
this would render our openjdk package unfit for many development scenarios.

If the jquery substitution is implemented directly in the javadoc tool
it must be limited to documentation aimed at lib*-java-doc packages
(either with an extra parameter passed by the helpers, or by detecting a
debian specific environment variable, like DEB_BUILD_ARCH).

Emmanuel Bourg

---

From: Markus Koschany <[email-masked]>
Date: Mon, 9 Jul 2018 23:29:57 +0200

> debian specific environment variable, like DEB_BUILD_ARCH).

We should really aim for the simplest solution. Actually I don't see any
need to patch the javadoc tool because we could easily solve this at the
packaging level. Just replace the embedded jquery library with symlinks
to Debian's system library and let openjdk depend on jquery. Problem solved.

Regards,

Markus

---

From: Emmanuel Bourg <[email-masked]>
Date: Mon, 9 Jul 2018 23:35:20 +0200

> to Debian's system library and let openjdk depend on jquery. Problem solved.

Sounds fine.

Or remove these mostly unused documentation packages we spend too much
time fixing or discussing after each new JDK hiccup ;)

---

From: Markus Koschany <[email-masked]>
Date: Mon, 9 Jul 2018 23:41:24 +0200

> time fixing or discussing after each new JDK hiccup ;)

Yeah, brutal but there is truth in it. I would really like to keep the
current level of support for Java documentation but I agree when it
becomes too painful and we waste too much time to fix such issues, we
should simply drop it.

Well, now we have to convince doku to implement this solution, or at
least to accept it, without closing the bug report again. Volunteers?

:E

---

From: tony mancill <[email-masked]>
Date: Mon, 9 Jul 2018 20:22:31 -0700

> becomes too painful and we waste too much time to fix such issues, we

I'm in favor of dropping the -java-doc packages completely and instead
using our time and effort to improve the state of our runtime libraries,
toolchain and application packages.  (It would be a different story if
we were developing a distribution for Java developers who don't have
ready access to other sources of documentation, but I have a hard time
imagining that our users would prefer javadocs over functioning and
secure libraries.)

> least to accept it, without closing the bug report again. Volunteers?

Hmm... I choose to believe that the bug (we're talking about [1],
right?) was mass-closed along with everything else that was open against
src:openjdk-9.  It seems like a reasonable and very "Debian" approach to
avoid embedding an available system library.  If we really want javadoc,
we could resubmit (preferably with a patch).

Cheers,
tony

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=883981

---

From: Markus Koschany <[email-masked]>
Date: Wed, 11 Jul 2018 18:35:21 +0200

Hi tony,

Am 10.07.2018 um 05:22 schrieb tony mancill:
[...]
> we could resubmit (preferably with a patch).

Yes, I was talking about #883981. There are two main issues with javadoc
at the moment. Firstly the syntax checker has been much more strict
since OpenJDK 8 and we currently work around a couple of problems by
simply ignoring javadoc errors, otherwise a lot more packages would be
RC buggy (FTBFS). Maybe this option will even go away in the future and
this would leave us with the following choice; either fix the underlying
error or drop the -doc package. Since we ship a lot of older or even
unmaintained software, which is still somehow useful to us though, I can
imagine that for some people our corresponding -doc packages are still a
good read because there is no equivalent source on the Internet (anymore).

Sure, that's not the sole reason why we should keep javadoc. My main
argument is that we would risk to overlook javadoc related issues in our
tool chain, if we dropped it completely. There should be at least a way
for people to create their own javadoc packages, preferably without too
much hassle. As long as that works, we could get over the rest. But
everything else is a regression.

And secondly then there is this jquery issue. I don't even know why they
need Javascript while HTML5 could do probably the same or even a better
job. Anyway we could fix this at the packaging level by replacing the
embedded copy with symlinks. There is one issue that remains: Should
-doc packages depend on jquery as well? There is a chance that the JRE
(which pulls in the jquery dependency) is not installed on the system.

So in my opinion it's not a choice between two options, javadoc yes or
no because at least for the jquery part this should be manageable.
However the first step is to acknowledge a problem but if bugs get
closed and everyone is more in favor of dropping javadoc completely,
then I also become rather "Meh".

Regards,

Markus

---

From: Martin Quinson <[email-masked]>
Date: Tue, 17 Jul 2018 01:15:20 +0200

Hello,

I'm building a package that provide some javadoc, so I got hit by that
bug myself too. The solution you propose (dropping javadoc packages)
does not exactly fit my needs, I must say ;)

Initially, I was simply looking for an example of d/rules chunk that I
could copy over to solve this in my package (and I'm still interested
in such an example).

But a better solution may be that you provide a tiny package providing
all what we need: the dependencies and all the links we need:

  /usr/share/javadoc/jquery/external/jquery/jquery.js -> /usr/share/javascript/jquery/jquery.js
  /usr/share/javadoc/jquery/jquery-1.10.2.js -> ??  no idea
  /usr/share/javadoc/jquery/jquery-ui.js -> /usr/share/javascript/jquery-ui/jquery-ui.js
  ...

That would greatly ease the patching of each and every javadoc
package: I'd just have to add a dependency and one single link:

  /usr/share/doc/simgrid/javadoc/jquery -> /usr/share/javadoc/jquery

But maybe that's just overengineered and I just need an example chunk,
I'm not sure. Could you point me to one of them?

In any case, thanks for your work and your help, that's really appreciated.
Mt

--
Pour ne plus rire jaune, voyez rouge.

---

From: tony mancill <[email-masked]>
Date: Mon, 16 Jul 2018 22:00:15 -0700

> then I also become rather "Meh".
 
Hi Markus,

Fair enough.  I can see the value in providing javadoc (or at least a
way to build the javadoc) for older versions of libraries. 

I think Martin Quinson's suggestion of "shim" jquery package has some
merit.  It means that we would have to touch every -java-doc package -
475 of them, by my current count - but I'm not sure that can be avoided
unless we take the path of patching openjdk-11 to use the Debian system
library.

And finally, although I'm still biased towards working on better runtime
support (to wit, libjide-oss-java is currently FTBFS, so the lintian
jquery warning seems less important than that), I don't think we should
ignore the problem and don't want anyone to feel unnecessarily "meh"
about it either... :)

Other ideas?

Cheers,
tony

---

From: Markus Koschany <[email-masked]>
Date: Tue, 17 Jul 2018 13:40:41 +0200

Hi tony,

Am 17.07.2018 um 07:00 schrieb tony mancill:
[...]
> unless we take the path of patching openjdk-11 to use the Debian system

I believe that every solution that involves patching all of our javadoc
packages is not a good one. :) Of course Martin can depend on Debian's
system jquery and use dh_link to replace the embedded copy with the one
installed on the system but I'm far too lazy too consider this a
worthwhile task for myself. It's not efficient, so to speak ;)

I'm in favor of tackling the issue at the root, openjdk-11. I will take
a closer look at DebConf18 and prepare a patch and resubmit my bug
report. Everything else is up to doko.

> ignore the problem and don't want anyone to feel unnecessarily "meh"

I agree there are a lot more interesting problems to work on but it's
far easier to solve than many of the other ones. As for
libjide-oss-java: I have reported the FTBFS months ago but it doesn't
look like that we will see a real solution soon. They depend on
functionality which was simply removed with OpenJDK 10. Fortunately only
some windows-specific classes are affected, so I could ultimately patch
them out and work around it. However we should strongly consider to ship
OpenJDK 8 with Buster. Then I could just build-depend on it and be done
for now and maybe in two years time there is a better solution. Actually
I don't see a reason why we couldn't do it, provided we mark OpenJDK 8
EOL security-wise and just use it for building/developing packages.

The most important problem is JavaFX at the moment because without that
libjide-oss-java is just another library. The reason why it was packaged
is mediathekview but without JavaFX is won't be part of Buster anyway
(and PDFsam, and Netbeans, and...)

Cheers,

Markus

---

From: Markus Koschany <[email-masked]>
Date: Tue, 17 Jul 2018 13:47:50 +0200

[...]

A quick solution is to depend on libjs-jquery and probably
libjs-jquery-ui and then use dh_link or debian/links and symlink the
offending Javascript files to the ones on the system.

Thus said I believe this should be fixed at the root. A good solution
would be to replace the embedded jquery library in OpenJDK-11 with a
symlink to the system library and maybe change some build-helper tools
like javahelper, maven-debian-helper etc. to automatically insert a
dependency on libjs-jquery and libjs-jquery-ui into already existing
substvars.

Regards,

Markus

---

From: Markus Koschany <[email-masked]>
Date: Sat, 11 Aug 2018 10:12:03 +0200

FTR: I have talked to Matthias Klose (doko) at DebConf18 about the
embedding of jquery into javadoc packages. He pointed me to a similar
discussion in doxygen which also embeds jquery while building doc packages.

In short he doesn't consider it to be a worthwhile task because there is
a risk of breaking the documentation when Debian's system jquery version
is either too old or too new. The security risk of embedding jquery is
also rather low in this case because the documentation is static in
contrast to web applications and it is unlikely that users would be
affected by jquery vulnerabilities.

README.jquery in doxygen explains the problem in more detail.

https://sources.debian.org/src/doxygen/1.8.13-10/debian/README.jquery/

All in all there is no chance that a patch to change the current
situation would be accepted, hence I no longer intend to spend time on it.

Markus

---

From: tony mancill <[email-masked]>
Date: Sat, 11 Aug 2018 11:12:32 -0700

Hi Markus,

I'm glad that you were able to discuss this directly with Matthias, and
thank you for sharing the gist of that conversation.  For our sanity, I
will take a look to see if we can get the severity of the lintian
warning [1] reduced to some lower level (pedantic?) or completely
ignored for javadoc packages.

Cheers,
tony

[1] https://lintian.debian.org/tags/embedded-javascript-library.html

---

From: Markus Koschany <[email-masked]>
Date: Sat, 11 Aug 2018 20:33:31 +0200

Hi tony,

Am 11.08.2018 um 20:12 schrieb tony mancill:
[...]
> [1] https://lintian.debian.org/tags/embedded-javascript-library.html

A severity of pedantic or informational would be appreciated but I guess
they won't change it. I just remembered that I build the documentation
in src:renpy from source with Sphinx and this tool also embeds jquery
and bootstrap. However Lintian still emits the warning about embedded
javascript libraries. We probably have to get used to the fact that we
will never be able to fix all Lintian warnings again (at least for some
packages).

Cheers,

Markus