Author Topic: Chauffeur BUG (Registering Listeners on specific hosts) (Possible FIX)  (Read 410 times)

TheKahn

  • Newbie
  • *
  • Posts: 15
  • Karma: +0/-0
Hi,

I've notice a terrible bug when I was trying to register listeners to a specific host in a multi-host setup.

As stated the registering process is done by adding the <clients></clients> tag into listeners.xml file. The specific host is given by  <clients>HOSTID:[host]</clients>. The problem is Chauffeur defines the host ID as the IP of the connected director with its current port.

The line responsible for this lies in the Host.java class

Code: [Select]
DirectorCommunicator director = new DirectorCommunicator("Host '"
+ chan.socket().getInetAddress().getHostAddress() + ":"
+ chan.socket().getPort() + "'", chan, getEnvironment());
   


The problem lies in the fact that the if a port range is not specified beforehand you will never know the correct hostID as a connection with the director is established anew with each run.



By changing the line into this.

Code: [Select]
DirectorCommunicator director = new DirectorCommunicator(this.getEnvironment().getHostId(), chan, getEnvironment());
The hostname of the host is registered as the hostID, which is static and will not change between runs.


I believe this might help some people!


Another thing, Chauffeur should really be able to let each host read its own config.xml file.
      

JeremyArnold

  • Moderator
  • Newbie
  • *****
  • Posts: 13
  • Karma: +1/-0
Thank you for pointing out this bug.  I believe I encountered a different symptom of the same underlying issue recently, and committed a fix for the next version of Chauffeur.  But my fix is a bit different than yours -- if you have a chance, perhaps you can try this and see if it resolves your issue.  In DirectorCommunicator.handshake(), replace:

Code: [Select]
msg.add(new HostInfo(getClientName(),
System.getenv(),
propMap,
environment.getRawDiscoveryData(),
environment.getHostHardwareThreads(),
environment.getLogicalCoreCount(),
environment.getPhysicalCoreCount(),
environment.getNumaNodeCount(),
environment.getMaxHeapSize(),
environment.getPhysicalMemorySize()));

with:
Code: [Select]
msg.add(new HostInfo(environment.getHostId(),
System.getenv(),
propMap,
environment.getRawDiscoveryData(),
environment.getHostHardwareThreads(),
environment.getLogicalCoreCount(),
environment.getPhysicalCoreCount(),
environment.getNumaNodeCount(),
environment.getMaxHeapSize(),
environment.getPhysicalMemorySize()));

It may also make sense to make the change you suggested.  The clientName string is mostly used for logging, and I think the IP/port were originally added to make that logging more useful.  But as long as we have that info somewhere in the logs, it should be sufficient to just write the host id.  I'll take a closer look and see what makes the most sense -- in particular, I want to check for any uses of the clientName outside of logging and make sure there's no code that is relying on the current naming.

I'll respond to your suggestion of having each host get its own config.xml in a separate reply.

JeremyArnold

  • Moderator
  • Newbie
  • *****
  • Posts: 13
  • Karma: +1/-0
Different hosts running different configs
« Reply #2 on: June 27, 2016, 02:42:44 PM »
Another thing, Chauffeur should really be able to let each host read its own config.xml file.

I agree there is a need to allow different hosts to do different things.  But I don't think having their own config.xml is the right approach.  Since the Director coordinates the work across all of the hosts, as well as the listeners on the Director (usually including power analyzer measurements and writing the results file), there are a number of areas where it is important for all of the hosts to stay in sync -- e.g. running the same number of load levels for the same amount of time.  (For use cases where we do not need this, perhaps a better approach would be to run multiple Director JVMs, each controlling one or more of the hosts.)  But while we need to keep the timing in sync, there are certainly interesting use cases for having different hosts running different load levels, or even different worklets.  And the same argument extends to having different clients on the same host running different load levels or different worklets.

At one point there was support for at least running a different series of load levels in each host or client (though they still all had to run the same worklet).  You can still see some of the code for this in AbstractDirectorSequence.  That support got broken with some other changes, and since we didn't have an immediate need for it at the time, I disabled it rather than fixing it.  Still haven't gotten around to re-implementing that, and when I do I'd really like to make it work for running different worklets as well.  So the implementation probably won't look the same as what is in AbstractDirectorSequence.

One of the challenges for implementing this is to come up with the right format in config.xml.  It may be helpful if we start by coming up with some sample scenarios where this feature would be useful, and then mocking up the config file to describe it.  Once we know what the configuration should look like, it should be relatively straightforward to pass the necessary information on to the Hosts and Clients.

If you have some other use case in mind which would not fit with the model I described, please let me know -- it's certainly possible that you're trying to do something that hasn't occurred to me.