[kolla][monasca] thresh keeps dying

Doug doug at stackhpc.com
Mon Jun 28 08:28:48 UTC 2021


On 16/06/2021 09:15, Tomasz Rutkowski wrote:
> can someone direct me where to search for the cause?

Have you seen this?

https://bugs.launchpad.net/kolla-ansible/+bug/1808805

The current behaviour without that fix on a multi-node cluster is for 
only one of the thresh containers to run in local mode and the others 
continually restart.

> as I understand failing to submit the topology to storm cluster
> shouldn't be final, and the process just dissapers there...
>
> + exec /opt/storm/bin/storm jar /monasca-thresh-source/monasca-thresh-stable-victoria/thresh/target/monasca-thresh-2.4.0-SNAPSHOT-shaded.jar -Djava.io.tmpdir=/var/lib/monasca-thresh/data monasca.thresh.ThresholdingEngine /etc/monasca/thresh-config.yml monasca-thresh
> Running: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/opt/storm -Dstorm.log.dir=/var/log/kolla/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /opt/storm/*:/opt/storm/lib/*:/opt/storm/extlib/*:/monasca-thresh-source/monasca-thresh-stable-victoria/thresh/target/monasca-thresh-2.4.0-SNAPSHOT-shaded.jar:/opt/storm/conf:/opt/storm/bin -Dstorm.jar=/monasca-thresh-source/monasca-thresh-stable-victoria/thresh/target/monasca-thresh-2.4.0-SNAPSHOT-shaded.jar -Dstorm.dependency.jars= -Dstorm.dependency.artifacts={} -Djava.io.tmpdir=/var/lib/monasca-thresh/data monasca.thresh.ThresholdingEngine /etc/monasca/thresh-config.yml monasca-thresh
> 687  [main] INFO  m.t.ThresholdingEngine - -------- Version Information --------
> 692  [main] INFO  m.t.ThresholdingEngine - monasca-thresh-2.4.0-SNAPSHOT-2021-06-06T08:32:08-${buildNumber}
> 693  [main] INFO  m.t.ThresholdingEngine - Instantiating ThresholdingEngine with config file: /etc/monasca/thresh-config.yml, topology: monasca-thresh
> 1000 [main] INFO  o.h.v.i.u.Version - HV000001: Hibernate Validator 5.2.1.Final
> 1197 [main] INFO  m.t.ThresholdingEngine - local set to false
> 1312 [main] INFO  m.t.i.t.MetricSpout - Created
> 1340 [main] INFO  m.t.i.t.EventSpout - EventSpout created
> 1516 [main] WARN  o.a.s.u.Utils - STORM-VERSION new 1.2.2 old null
> 1516 [main] INFO  m.t.ThresholdingEngine - submitting topology monasca-thresh to non-local storm cluster
> 1549 [main] INFO  o.a.s.StormSubmitter - Generated ZooKeeper secret payload for MD5-digest: -7012431400424907995:-9108807134284416946
> 1728 [main] INFO  o.a.s.u.NimbusClient - Found leader nimbus : storm1:6627
> 1751 [main] INFO  o.a.s.s.a.AuthUtils - Got AutoCreds []
> 1756 [main] INFO  o.a.s.u.NimbusClient - Found leader nimbus : storm1:6627
> Exception in thread "main" java.lang.RuntimeException: Topology with name `monasca-thresh` already exists on cluster
>          at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:237)
>          at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:387)
>          at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:159)
>          at monasca.thresh.ThresholdingEngine.run(ThresholdingEngine.java:111)
>          at monasca.thresh.ThresholdingEngine.main(ThresholdingEngine.java:82)
>
>
> regards



More information about the openstack-discuss mailing list