Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added VNF resource consumption functions #78

Merged
merged 11 commits into from
Aug 22, 2019

Conversation

ldklenner
Copy link
Contributor

VNF/SF can no specifiy a resource function which defines the needed capacity dependent on the individual SF load. If no resource function is specified, the default function f(x)=x will be used. When using resource functions a path parameter must be passed to the simulator.init function.

 a:
    processing_delay_mean: 5.0
    processing_delay_stdev: 0.0
    resource_function_id: A

The reader when constructing the sf_list will try to load "A.py" in the specified path.
"A.py" has to contain a resource_function(load) function.

Note: do not include ".py" in the yaml resource_function_id field.
Note: resource functions are optional. Passing no path to the simulator will assign the default function to all SFs with warnings.

@ldklenner
Copy link
Contributor Author

ldklenner commented Aug 21, 2019

Local tests work and produced the intended output. Travis tests encounter severe problems, do to the fact that "test_simulator.py" do not use the simulator interface function simulator.apply.

simulator.apply is used to construct the list of available SF at each node from the placement parameter. Without the function call the list stays empty. The flowsimulator when attempting to record the current load of a SF will fail as the SF is placed but not listed as available.

@ldklenner
Copy link
Contributor Author

ldklenner commented Aug 21, 2019

SF load is now recorded from the beginning. Next commit will change travis.yaml to explicit load resource funtions.

Copy link
Member

@stefanbschneider stefanbschneider left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! Is this ready to merge?

@qarawlus Please have a look at the changes, so you are aware 👍

src/coordsim/simulation/flowsimulator.py Show resolved Hide resolved
@ldklenner
Copy link
Contributor Author

Now it should be ready to merge.

@stefanbschneider stefanbschneider merged commit a51353c into RealVNF:master Aug 22, 2019
@stefanbschneider
Copy link
Member

Closes #54

for node_id, placed_sf_list in actions.placement.items():
available_sf = {}
for sf in placed_sf_list:
available_sf[sf] = self.simulator.params.network.nodes[node_id]['available_sf'].get(sf, {'load': 0.0})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this do exactly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each node stores a list of SFs that where placed and there current load as a node attribute. When changing the placement the list has to be updated.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the .get(sf, {'load': 0.0}) gets the current load of the SF at the node and defaults to 0 if the SF wasn't available?

@stefanbschneider
Copy link
Member

When testing the changes with our RL algorithm, I ran into the following error:

File "c:\users\stefan\git-repos\work\swc\coordination-simulation\src\coordsim\simulation\flowsimulator.py", line 241, in process_flow
    self.params.network.nodes[current_node_id]['available_sf'][sf]['load'] -= flow.dr
KeyError: 'c'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "c:\users\stefan\git-repos\work\swc\coordination-simulation\src\coordsim\simulation\flowsimulator.py", line 120, in pass_flow
    yield self.env.process(self.process_flow(flow, sfc))
KeyError: 'c'

Seems like there's something wrong with the available SF at each node. I'm going through the code to better understand what's the issue right now. Thankful for any tips you might have.

@stefanbschneider
Copy link
Member

I think I have the error:

When reducing the load at an SF because a flow is done processing, self.params.network.nodes[current_node_id]['available_sf'][sf]['load'] -= flow.dr
is called: https://github.com/RealVNF/coordination-simulation/blob/master/src/coordsim/simulation/flowsimulator.py#L241

It's possible that the placement changes in between starting and stopping to process a flow and that at the time of accessing the dict, the SF is no longer available at the node.

@ldklenner
Copy link
Contributor Author

It's possible that the placement changes in between starting and stopping to process a flow and that at the time of accessing the dict, the SF is no longer available at the node.

Yes, i think that is exactly what happens. In the previous version a SF could be removed while a flow is still in process without violating capacity demands, the capacity when finished processing would simply increased. With this version the load has to be recorded precisely.

@stefanbschneider
Copy link
Member

Yep. I'm not sure what would be the best way to handle this.

  • a) Either, we're strict and drop any flows that are being processed when a VNF is moved
  • b) Or we stick to the previous model and allow flows to finish processing

@stefanbschneider
Copy link
Member

stefanbschneider commented Aug 22, 2019

a) Drop flows when VNF is removed

  • Would be easy to implement
  • But would basically always lead to dropped flows when changing the placement
  • Not necessarily realistic

b) Allow flows to finish processing

  • Could argue that it's more realistic to finish VNF "gracefully", ie, don't send any new flows but allow active flows to finish before stopping VNF
  • But then we should still account for the resource consumption of the VNF while it's still processing
  • Not sure how to do this without too much effort. Currently we only keep track of available SFs, which makes sense.
  • Of course, we could also simply ignore the resource consumption of the SF in this case, since now new flows will be scheduled to it anyway. But not 100% accurate

Any thoughts or ideas?

@stefanbschneider
Copy link
Member

stefanbschneider commented Aug 22, 2019

I need to get rid of the error, so my current fix goes with b) and allows flows to finish processing when an SF is removed. #80

But it doesn't account for resource consumption of these (almost) removed VNFs and simply ignores it. Not perfect, but hopefully not a big issue - should keep it in mind anyways. This is in line with what we had previously.

Any better solution is more than welcome

@ldklenner
Copy link
Contributor Author

I think option a) would lead to more serious consequences especially when a flow is simultaneously processed by multiple VNFs.

Option b) is the way to go. The primary decision if a flow gets processed is based on the placement. If a SF is no longer included in the placement a flow will not be processed by it.
What we could do is: keep processing SFs in the available SF list even if there no longer included in the placement. When the load is decreased to 0 and the SF is no longer in the placement, the SF will be removed from the available list.

@stefanbschneider
Copy link
Member

What we could do is: keep processing SFs in the available SF list even if there no longer included in the placement. When the load is decreased to 0 and the SF is no longer in the placement, the SF will be removed from the available list.

Yes, that's also what I thought. Not sure how much effort this would be?

@ldklenner ldklenner deleted the dev-resource-functions branch August 22, 2019 15:18
@ldklenner
Copy link
Contributor Author

I'm looking into it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants