Open Stack

On 07/24/2015 12:00 PM, Tang Chen wrote:
>
> On 07/24/2015 10:08 AM, Tang Chen wrote:
>>
>> On 07/23/2015 11:44 PM, Asselin, Ramy wrote:
>>> Are you running on 'master' nodes? I remember seeing an issue where 
>>> with a recent version of Jenkins or a plugin where it doesn't 
>>> execute jobs on the master node.
>>> But when run on non-master jenkins slaves, it works fine.
>>
>> I checked my configuration, and made sure these things:
>> 1. I have only a master node, no slave node.
>> 2. I have 20 idle executors on master node.
>> 3. My master node is online.
>> 4. My master node is set to "Utilize this node as much as possible".
>> 5. zuul is able to be notified by Gerrit, and tell Jenkins to start 
>> jobs.
>>
>> But the jobs are always pending.
>
> And my Gearman reports this error sometimes.
>
> 2015-07-25 10:50:44,914 ERROR gear.Server: Exception in poll loop:
> Traceback (most recent call last):
>   File "/usr/local/lib/python2.7/dist-packages/gear/__init__.py", line 
> 2614, in _doPollLoop
>     self._pollLoop()
>   File "/usr/local/lib/python2.7/dist-packages/gear/__init__.py", line 
> 2626, in _pollLoop
>     ret = self.poll.poll()
> IOError: [Errno 4] Interrupted system call
>
> Not sure if it has anything to do with this problem.
>
> In Jenkins GUI, Gearman connection is tested successfully on 
> 127.0.0.1:4730.

Seeing from zuul debug log, Gearman has successfully submitted the jobs.

2015-07-25 11:42:09,255 DEBUG zuul.Scheduler: Adding trigger event: 
<TriggerEvent patchset-created openstack-dev/sandbox master 205360,1>
2015-07-25 11:42:09,256 DEBUG zuul.Scheduler: Done adding trigger event: 
<TriggerEvent patchset-created openstack-dev/sandbox master 205360,1>
2015-07-25 11:42:09,256 DEBUG zuul.Scheduler: Run handler awake
2015-07-25 11:42:09,256 DEBUG zuul.Scheduler: Fetching trigger event
2015-07-25 11:42:09,256 DEBUG zuul.Scheduler: Processing trigger event 
<TriggerEvent patchset-created openstack-dev/sandbox master 205360,1>
2015-07-25 11:42:09,257 DEBUG zuul.IndependentPipelineManager: Event 
<TriggerEvent patchset-created openstack-dev/sandbox master 205360,1> 
for change <Change 0x7ff518312c10 205360,1> matched <EventFilter types: 
patchset-created> in pipeline <IndependentPipelineManager check>
2015-07-25 11:42:09,257 INFO zuul.Scheduler: Adding 
openstack-dev/sandbox, <Change 0x7ff518312c10 205360,1> to <Pipeline check>
2015-07-25 11:42:09,257 DEBUG zuul.IndependentPipelineManager: 
Considering adding change <Change 0x7ff518312c10 205360,1>
2015-07-25 11:42:09,257 DEBUG zuul.IndependentPipelineManager: Checking 
for changes needed by <Change 0x7ff518312c10 205360,1>:
2015-07-25 11:42:09,257 DEBUG zuul.IndependentPipelineManager:   No 
changes needed
2015-07-25 11:42:09,257 DEBUG zuul.IndependentPipelineManager: Adding 
change <Change 0x7ff518312c10 205360,1> to queue <ChangeQueue check: 
openstack-dev/sandbox>
2015-07-25 11:42:09,258 DEBUG zuul.IndependentPipelineManager: Event 
<TriggerEvent patchset-created openstack-dev/sandbox master 205360,1> 
for change <Change 0x7ff518312c10 205360,1> matched <EventFilter types: 
patchset-created> in pipeline <IndependentPipelineManager silent>
2015-07-25 11:42:09,258 INFO zuul.Scheduler: Adding 
openstack-dev/sandbox, <Change 0x7ff518312c10 205360,1> to <Pipeline silent>
2015-07-25 11:42:09,258 DEBUG zuul.IndependentPipelineManager: 
Considering adding change <Change 0x7ff518312c10 205360,1>
2015-07-25 11:42:09,258 DEBUG zuul.IndependentPipelineManager: Unable to 
find change queue for change <Change 0x7ff518312c10 205360,1> in project 
openstack-dev/sandbox
2015-07-25 11:42:09,306 DEBUG zuul.IndependentPipelineManager: Starting 
queue processor: check
2015-07-25 11:42:09,306 DEBUG zuul.IndependentPipelineManager: Checking 
for changes needed by <Change 0x7ff518312c10 205360,1>:
2015-07-25 11:42:09,306 DEBUG zuul.IndependentPipelineManager:   No 
changes needed
2015-07-25 11:42:09,306 DEBUG zuul.IndependentPipelineManager: Preparing 
ref for: <Change 0x7ff518312c10 205360,1>
2015-07-25 11:42:09,307 INFO zuul.IndependentPipelineManager: Change 
<Change 0x7ff518312c10 205360,1> depends on changes []
2015-07-25 11:42:09,307 DEBUG zuul.MergeClient: Submitting job <gear.Job 
0x7ff518325490 handle: None name: merger:merge unique: 
b425daae0cec4ff3b0d4920ee5533c9c> with data {'items': [{'oldrev': None, 
'newrev': None, 'refspec': u'refs/changes/60/205360/1', 'merge_mode': 2, 
'number': u'205360', 'project': 'openstack-dev/sandbox', 'url': 
'ssh://Fnst@review.openstack.org:29418/openstack-dev/sandbox', 'branch': 
u'master', 'patchset': u'1', 'ref': 'Z81183af13086459d9d9fb8c06af86c09'}]}
2015-07-25 11:42:09,308 DEBUG zuul.IndependentPipelineManager: Finished 
queue processor: check (changed: False)
2015-07-25 11:42:09,309 DEBUG zuul.IndependentPipelineManager: Starting 
queue processor: silent
2015-07-25 11:42:09,309 DEBUG zuul.IndependentPipelineManager: Finished 
queue processor: silent (changed: False)
2015-07-25 11:42:09,309 DEBUG zuul.IndependentPipelineManager: Starting 
queue processor: patch
2015-07-25 11:42:09,309 DEBUG zuul.IndependentPipelineManager: Finished 
queue processor: patch (changed: False)
2015-07-25 11:42:09,309 DEBUG zuul.Scheduler: Run handler sleeping

Thanks. :)

>
> Thanks.
>
>>
>> I'm now trying to setup a slave node and try again. :)
>>
>> And anyone has any idea of this, please let me know.
>>
>> Thanks.
>>
>>> -----Original Message-----
>>> From: Tang Chen [mailto:tangchen at cn.fujitsu.com]
>>> Sent: Thursday, July 23, 2015 12:38 AM
>>> To: OpenStack Development Mailing List (not for usage questions)
>>> Subject: Re: [openstack-dev] [CI] Jenkins jobs are not executed when 
>>> setting up a new CI system.
>>>
>>> Hi, all
>>>
>>> The Jenkins jobs submitted by zuul are always "pending-Waiting for 
>>> next available executor" .
>>>
>>> And Jenkins log shows the following:
>>>
>>> Jul 24, 2015 11:09:04 AM hudson.plugins.gearman.MyGearmanWorkerImpl
>>> submitFunction
>>> WARNING: ---- Worker _exec-0 exception while executing function 
>>> hudson.plugins.gearman.StartJobWorker
>>> java.lang.RuntimeException: java.lang.RuntimeException:
>>> java.util.concurrent.CancellationException
>>> at
>>> org.gearman.worker.AbstractGearmanFunction.call(AbstractGearmanFunction.java:136) 
>>>
>>> at
>>> org.gearman.worker.AbstractGearmanFunction.call(AbstractGearmanFunction.java:22) 
>>>
>>> at
>>> hudson.plugins.gearman.MyGearmanWorkerImpl.submitFunction(MyGearmanWorkerImpl.java:590) 
>>>
>>> at
>>> hudson.plugins.gearman.MyGearmanWorkerImpl.work(MyGearmanWorkerImpl.java:374) 
>>>
>>> at
>>> hudson.plugins.gearman.AbstractWorkerThread.run(AbstractWorkerThread.java:166) 
>>>
>>> at java.lang.Thread.run(Thread.java:745)
>>> Caused by: java.lang.RuntimeException:
>>> java.util.concurrent.CancellationException
>>> at
>>> hudson.plugins.gearman.StartJobWorker.executeFunction(StartJobWorker.java:116) 
>>>
>>> at
>>> org.gearman.worker.AbstractGearmanFunction.call(AbstractGearmanFunction.java:125) 
>>>
>>> ... 5 more
>>> Caused by: java.util.concurrent.CancellationException
>>> at hudson.remoting.AsyncFutureImpl.get(AsyncFutureImpl.java:77)
>>> at
>>> hudson.plugins.gearman.StartJobWorker.safeExecuteFunction(StartJobWorker.java:196) 
>>>
>>> at
>>> hudson.plugins.gearman.StartJobWorker.executeFunction(StartJobWorker.java:114) 
>>>
>>> ... 6 more
>>>
>>> Jul 24, 2015 11:09:04 AM hudson.plugins.gearman.MyGearmanWorkerImpl 
>>> work
>>> INFO: ---- Worker _exec-0 sending initial grab job Jul 24, 2015 
>>> 11:09:04 AM hudson.plugins.gearman.MyGearmanWorkerImpl
>>> handleSessionEvent
>>> INFO: ---- Worker _exec-1 received unique job assignment Jul 24, 
>>> 2015 11:09:04 AM hudson.plugins.gearman.MyGearmanWorkerImpl work
>>> INFO: ---- Worker _exec-1 executing function Jul 24, 2015 11:09:04 
>>> AM hudson.plugins.gearman.StartJobWorker
>>> safeExecuteFunction
>>> INFO: ---- Worker _exec-1 scheduling devstack-vm-test build #14 on 
>>> with UUID 83d4fd10c1ad4f1ead51beb2adf23ccd and build params
>>> [(StringParameterValue) BASE_LOG_PATH='03/204803/1',
>>> (StringParameterValue) ZUUL_PIPELINE='check', (StringParameterValue) 
>>> ZUUL_UUID='83d4fd10c1ad4f1ead51beb2adf23ccd', (StringParameterValue) 
>>> LOG_PATH='03/204803/1/check/devstack-vm-test/83d4fd1',
>>> (StringParameterValue) ZUUL_CHANGE_IDS='204803,1',
>>> (StringParameterValue) ZUUL_PATCHSET='1', (StringParameterValue) 
>>> ZUUL_BRANCH='master', (StringParameterValue) 
>>> ZUUL_REF='refs/zuul/master/Z07d022076a68448d842bd1a47dd42e19',
>>> (StringParameterValue)
>>> ZUUL_COMMIT='174cac545549f086e07f32edbe34b70c4155a7fc',
>>> (StringParameterValue) ZUUL_URL='http://10.124.196.205/p/',
>>> (StringParameterValue) ZUUL_CHANGE='204803', (StringParameterValue) 
>>> ZUUL_CHANGES='openstack-dev/sandbox:master:refs/changes/03/204803/1',
>>> (StringParameterValue) ZUUL_PROJECT='openstack-dev/sandbox']
>>>
>>>
>>> It seems that Gearman doesn't work properly.
>>>
>>> Do you guys have any idea of this ?
>>>
>>> Thanks.
>>>
>>> __________________________________________________________________________ 
>>>
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: 
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>> __________________________________________________________________________ 
>>>
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: 
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> .
>>>
>>
>>
>> __________________________________________________________________________ 
>>
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: 
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> .
>>
>
>
> __________________________________________________________________________ 
>
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: 
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> .
>