Author Topic: Running bash scripts as transaction behaviour  (Read 25936 times)

TheKahn

  • Newbie
  • *
  • Posts: 15
  • Karma: +0/-1
Re: Running bash scripts as transaction behaviour
« Reply #15 on: January 14, 2016, 04:49:25 AM »
I'm glad you found a solution.

Listeners are intentionally designed to be passive, and not be able to influence the behavior of the worklets.  I'm not inclined to change that behavior.  I'd prefer to use more explicit control mechanisms for cases like this. 


It is fine with me. But you need to grant those listners access to more variables and of course the transaction instance itself.

Quote
For the default listeners -- I was thinking you were referring to the RawFileWriter, but now I understand you meant the WriterRunListener, which writes run progress to System.out.  This is currently hard-coded (e.g. ClientJvm.main()).  That could be commented out, but perhaps the better solution would be to update the WriterRunListener implementation to avoid the exception.  Another option to consider would be to store your custom results in Metrics rather than returning it as the score, since Chauffeur pretty much assumes each interval has a single result for the interval.


Actually I found a nice way to prevent the nullPointers and have my own listener. I just keep the score variable.  Only the client logs and host logs will continute to have both outputs. Why did you not register the client and host listener variant in the listener.xml?


I am using both variants (writing metrics into the results.xml and via the runWriterListener onto the screen). At the moment I am again fighting with the listener, as it is not possible to pass metrics to the RawFileWriter after the measurementComplete phase and listeners cannot access result at this point of time, which I find strange because the score was already calculated at this point.

Your listeners definitely need to have access to the scores at this point of time.

Quote
For AbstractInterval, I would lean towards keeping getIntervalDef protected, but adding methods to the Interval interface to return information from the IntervalDef where appropriate (similar to the current getName() method).  But I could be convinced otherwise.  Can you remind me what information you are getting from the IntervalDef in your code?

Sure, I use it to get the ScenarioMixID and the TransactionParameterMap.


Another important question. How can I pause the PowerAnalyzer measurment from the transaction method and restart it from within?


Cheers

JeremyArnold

  • Moderator
  • Newbie
  • *****
  • Posts: 15
  • Karma: +1/-0
Re: Running bash scripts as transaction behaviour
« Reply #16 on: January 14, 2016, 02:34:33 PM »
You are correct that the WriterRunListener should probably be registered in listeners.xml rather than hard-coded.  I've added this to our list for a future release.
As you know, there are several listener notifications for each interval, in the sequence:
  • intervalStarting
  • preMeasurementStarting
  • measurementStarting
  • measurementEnded
  • postMeasurementEnded
  • intervalEnding
  • intervalEnded
.
The interval scores are calculated between the measurementEnded and postMeasurementEnded notifications.  The RawFileWriter records the results (including metrics) in intervalEnded.  So I believe you should be able to record your metrics in postMeasurementEnded.  At that point you'll have access to the IntervalResults, and any metrics that get stored here should be included in results.xml by the RawFileWriter.

It is not possible to start and stop power measurements within a transaction.  This really doesn't fit with the design of Chauffeur.  In particular, for the typical Chauffeur use case, many transactions may be running concurrently, so it wouldn't make sense to allow individual transactions to control the power measurement.  Instead, performance and power measurements are made on a per-interval basis.  So if you have a requirement to start and stop power measurements during the run, I think our goal would be to determine how to make each of these measurements into its own interval rather than a transaction.

TheKahn

  • Newbie
  • *
  • Posts: 15
  • Karma: +0/-1
Re: Running bash scripts as transaction behaviour
« Reply #17 on: January 14, 2016, 03:36:01 PM »
You are correct that the WriterRunListener should probably be registered in listeners.xml rather than hard-coded.  I've added this to our list for a future release.
As you know, there are several listener notifications for each interval, in the sequence:
  • intervalStarting
  • preMeasurementStarting
  • measurementStarting
  • measurementEnded
  • postMeasurementEnded
  • intervalEnding
  • intervalEnded
.
The interval scores are calculated between the measurementEnded and postMeasurementEnded notifications.  The RawFileWriter records the results (including metrics) in intervalEnded.  So I believe you should be able to record your metrics in postMeasurementEnded.  At that point you'll have access to the IntervalResults, and any metrics that get stored here should be included in results.xml by the RawFileWriter.

Yes, I get the results but unfortunately my metrics are not written into the results.xml if I do not define and add them in the measurementComplete method of my listener. For some reasons.


Quote
It is not possible to start and stop power measurements within a transaction.  This really doesn't fit with the design of Chauffeur.  In particular, for the typical Chauffeur use case, many transactions may be running concurrently, so it wouldn't make sense to allow individual transactions to control the power measurement.  Instead, performance and power measurements are made on a per-interval basis.  So if you have a requirement to start and stop power measurements during the run, I think our goal would be to determine how to make each of these measurements into its own interval rather than a transaction.

OK, good to know that. Well, just to let you know, I am talking about an overhead of around 10 samples for a measurement duration of 200 seconds.


Btw. There seems to be a bug in Chauffeur, which prevents you from changing the worklet context in the init method of the transaction. It causes the JVM to abort I will recreate this bug and post the host output (the error originates in the host not the client) later.



The code is not the real code but just exemplary of course :)


Code: [Select]
public void init(IntervalContext<? extends U> context) throws InvalidConfigurationException {

//DO SOME OTHER STUFF

//FAILS WHEN I DO THIS
context.getWorklet().getWorkletDefinition().getParameters().replace("key", "new ConfigObject")


super.init(context);
}
« Last Edit: January 14, 2016, 03:51:39 PM by TheKahn »

TheKahn

  • Newbie
  • *
  • Posts: 15
  • Karma: +0/-1
Re: Running bash scripts as transaction behaviour
« Reply #18 on: January 19, 2016, 11:58:42 AM »
Hi, I have antoher problem, which is really painful and very urgent. If my code is correct then this must be a bug in Chauffuer. I would really appreciate it if you could take a look.



This code works only for the measurementEnded event. According to you the metrics in the result.xml should be overwritten if the RawFileWriter takes the postMeasurementPhase also into consideration. However for me it does not work. Do I have an error in my code or is it a bug in Chaufeur?

The listener ist only registerd to the client.

Code: [Select]
package org.spec.chauffeur.energyEfficiencyMetrics.Listeners;



@SuppressWarnings("unchecked")
public class EnergyEfficiencyMetricsResultListener extends RunListenerAdapter {


private static final MetricGroup GROUP_ROOT = MetricGroup.createRootGroup("energy_efficiency_metrics");
private static final MetricGroup GROUP_SCENARIO = GROUP_ROOT.createSubgroup("scenario");
private static final MetricGroup GROUP_SCENARIO_INFORMATION = GROUP_SCENARIO.createSubgroup("information");


private static final Metric<String>[] FIREWALL_SCENARIO_SCENARIO_DATA = (Metric<String>[]) new Metric<?>[] {
GROUP_SCENARIO_INFORMATION.createMetric("scenarioName"),
GROUP_SCENARIO_INFORMATION.createMetric("useCase"), GROUP_SCENARIO_INFORMATION.createMetric("metricType"),
GROUP_SCENARIO_INFORMATION.createMetric("workloadInPercent"),
GROUP_SCENARIO_INFORMATION.createMetric("amountOfIterations"),
GROUP_SCENARIO_INFORMATION.createMetric("measurementDuration"),
GROUP_SCENARIO_INFORMATION.createMetric("nicLimit") };

private MetricsProvider provider;

@Override
public void suiteStarted(SuiteEvent evt) {
provider = new MetricsProvider("energy_efficiency_metrics");
}

@Override
public void measurementEnded(IntervalEvent<?> evt) throws ListenerException {


Metrics metrics = evt.getContext().getMetrics();
MetricData metricData = metrics.getMetricData(provider);

metricData.set(GROUP_SCENARIO_INTERVALNAME, evt.getContext().getInterval().getName());

ArrayList<String> scenarioMetrics = new ArrayList<String>();
scenarioMetrics.add("test");
scenarioMetrics.add("test");
scenarioMetrics.add("test");
scenarioMetrics.add("test");
scenarioMetrics.add("test");

for (int i = 0; i < FIREWALL_SCENARIO_SCENARIO_DATA.length; i++) {
metricData.set(FIREWALL_SCENARIO_SCENARIO_DATA[i], scenarioMetrics.get(i));
}


}
}
}

// TEST
@Override
public void postMeasurementEnded(IntervalEvent<?> evt) throws ListenerException {
Metrics metrics = evt.getContext().getMetrics();
MetricData metricData = metrics.getMetricData(provider);

metricData.set(GROUP_SCENARIO_INTERVALNAME, evt.getContext().getInterval().getName());

ArrayList<String> scenarioMetrics = new ArrayList<String>();
scenarioMetrics.add("test2");
scenarioMetrics.add("test2");
scenarioMetrics.add("test2");
scenarioMetrics.add("test2");
scenarioMetrics.add("test2");
scenarioMetrics.add("test2");
scenarioMetrics.add("test2");

for (int i = 0; i < FIREWALL_SCENARIO_SCENARIO_DATA.length; i++) {
metricData.set(FIREWALL_SCENARIO_SCENARIO_DATA[i], scenarioMetrics.get(i));
}
}

}





Btw. Is there a possibility to set a configuration parameter to run the suite n-times?


Thank you very much in advance.

Cheers

« Last Edit: January 19, 2016, 04:15:41 PM by TheKahn »

TheKahn

  • Newbie
  • *
  • Posts: 15
  • Karma: +0/-1
Re: Running bash scripts as transaction behaviour
« Reply #19 on: February 02, 2016, 11:31:28 AM »
I know the cause of this problem:

In my last version I registerd my listener to the clients. Now, after registering my listener to the director and only the directory. The director writes the results into the XML file also during the postMeasurementEnded and IntervalEnding phases.