Draft: Tobi Dance Branch #12

frehe · 2022-12-14T13:59:28Z

No description provided.

…mass M to m_cart and cart pole mass m to m_pole added comments about meaning of "stage" in MPPI renamed TargetPositionGenerator to TargetTrajectoryGenerator, since we target a complete trajectory, not just cart position. renamed my_logger() to get_logger()

…zer config changes to updated_attributes before executing optimizer.step

…istency with other config files. added more docstrings for undocumented methods and constructors. added updated_attributes to barebones controller. improved formatting of logging output added dictdiffer to requirements.txt renamed target_position cost function to cartpole_trajectory_cost.py. replaced some load_config with the load_or_reload_config_if_modified, so that the configs are cached for change checking. added logger to Compile.py

…g and the cartpole is at least balancing itself and swinging up from down position. Big changes all over! Now anytime any of the config files is modified during runtime, the using class has its tf.variable assigned the new value. To get this work, the variable MUST NOT EXIST before it is set this way the first time. Otherwise the compiler just uses the field at compile time and the changes are never seen inside. The cartpole window now does not reset the initial position and angle sliders on each start.

…hanging the cost weights during runtime! Yay, finally! Trick was to make sure that all values are actually propagated to tf variables, e.g. numpy arrays, ints, string, not just float Changed logging format to start with level for better readability. added string type to computation_library.py

…nfig values in cartpole_trajectory_generator.py. added dt to params in cartpole_trajectory_generator.step() call.

…on and equilibrium from updated attributes before they were updated in controller

…xtrapolate it, and added option for using terminal cost or entire trajectory cost in cartpole_trajectory_cost.py. added more params to config_cost_functions.yml. fiddled with logging colors. changed yaml loader to one that can deal with scientific format numbers (ruamel.yaml)

frehe · 2022-12-14T14:03:54Z

Controllers/__init__.py

@@ -26,7 +27,7 @@
 class template_controller(ABC):
    _has_optimizer = False
    # Define the computation library in your controller class or in the controller's configuration:
-    _computation_library: "type[ComputationLibrary]" = None


Technically, we always work with a static reference to the computation library. That means we don't create instances of it, but always call the methods on the class itself. Therefore, the type of this attribute is type. This is really only for documentation. The code will work no matter what we put here.

Btw, the subscript type notation "type[ComputationLibrary]" in quotes is necessary for python<=3.8. In later versions one can use type[ComputationLibrary], but we maintain backward compatibility for now

frehe · 2022-12-14T14:05:29Z

Cost_Functions/__init__.py

    # Default: Class supports all libs to compute costs
    supported_computation_libraries = {NumpyLibrary, TensorFlowLibrary, PyTorchLibrary}

-    def __init__(self, controller: template_controller, ComputationLib: "type[ComputationLibrary]") -> None:
-        self.controller = controller
+    def __init__(self, controller: template_controller, ComputationLib: "type[ComputationLibrary]", config:dict=None) -> None:


Suggested change

def __init__(self, controller: template_controller, ComputationLib: "type[ComputationLibrary]", config:dict=None) -> None:

def __init__(self, controller: template_controller, ComputationLib: "type[ComputationLibrary]", config:"Optional[dict]"=None) -> None:

frehe · 2022-12-14T14:10:49Z

Cost_Functions/cost_function_wrapper.py


    def configure(
        self,
        controller: template_controller,
        cost_function_specification=None,
    ):
+        """
+        Configures the cost function. TODO This lazy constructor is needed why?


I agree actually. At this point I don't see a reason for the wrapper any(more).

The only justification was: You can create a cost function wrapper without knowing at that time which cost function (default, trajectory follower, boundary grad, etc.) to use.

However, the only location where this object is instantiated, is in the controller constructor. There, the cost function information is available.

frehe · 2022-12-14T14:12:29Z

Cost_Functions/cost_function_wrapper.py

@@ -63,7 +78,7 @@ def get_stage_cost(self, states: TensorType, inputs: TensorType, previous_input:
        return self.cost_function.get_stage_cost(states, inputs, previous_input)

    def get_trajectory_cost(
-        self, state_horizon: TensorType, inputs: TensorType, previous_input: TensorType = None
+        self, state_horizon: TensorType, inputs: TensorType, previous_input: TensorType = None, config:dict=None


Suggested change

self, state_horizon: TensorType, inputs: TensorType, previous_input: TensorType = None, config:dict=None

self, state_horizon: TensorType, inputs: TensorType, previous_input: TensorType = None, config:"Optional[dict]"=None

frehe · 2022-12-14T14:13:00Z

Cost_Functions/cost_function_wrapper.py

@@ -1,32 +1,41 @@
 from importlib import import_module
 import os
+
+from Control_Toolkit.others.globals_and_utils import get_logger
 from SI_Toolkit.computation_library import TensorType
 import yaml
 from copy import deepcopy as dcp
 from types import MappingProxyType


Suggested change

from types import MappingProxyType

from types import MappingProxyType

from typing import Optional

frehe · 2022-12-14T14:15:43Z

requirements.txt

Suggested change

gym

gymnasium

Package name changed

added prefs to cartpole gui for initial position and angle increased num_rollouts to 1000, decreased horizon to 25 time steps, changed cost function for spin to pure cart angle speed. Now spin, balance, and shimmy all work quite well. moved cartpole_trajectory_generator back out outside compiled TF code for ease of development and debugging, it is cheap so this is OK. added more logging of what gets compiled. added time to arguments for the base controller so that contained methods can access current time. added ruamel.yaml to requirements (this is the yaml parser that can handle scientific notation numbers)

…viors).

# Conflicts: # Control_Toolkit_ASF_Template/Cost_Functions/EnvironmentName/cost_function_barebone.py # Control_Toolkit_ASF_Template/config_optimizers.yml # Controllers/__init__.py # Controllers/controller_neural_imitator_tf.py # Cost_Functions/__init__.py # Cost_Functions/cost_function_wrapper.py # Optimizers/optimizer_mppi.py # Optimizers/optimizer_rpgd_me_tf.py # Optimizers/optimizer_rpgd_ml_tf.py # Optimizers/optimizer_rpgd_particle_tf.py # Optimizers/optimizer_rpgd_tf.py # requirements.txt

Started merging from remote/origin/Tobi_Dance

# Conflicts: # Controllers/__init__.py # Controllers/controller_mpc.py # Cost_Functions/cost_function_wrapper.py # Optimizers/optimizer_rpgd_tf.py # others/environment.py # others/globals_and_utils.py

fixed logger->log renaming

…cal-cartpole repo. Fix some cartpole parameter import. Fix rename of config_cost_functions.yml. Add timer.py and yes_or_no.py utility classes.

… seem to work. mppi parameters copied from development. cost weights adjusted. controller update interval inccreased to 25ms from 20ms to match simulator and actual achieved rate with dancer mppi. mppi rollouts reduced to 700 to speed up control. PhysicalCartPoleDriver.py has class self variable for reference from other classes to the instance. Compile.py raises exception when either option is undefined.

…ome damping. But balancing still not working very well. Cannot follow cart position well. gets into endless repeats of trying to swing up. Fixed increment and dec of target position. Reduced control period back to 20ms from 25ms. added a controller reset to controller_mpc.py that stops the dance song. added a return to center function in PhysicalCartPoleDriver.py with key "C"

setting up to measure model mismatch in controller_mpc.py, not implemented yet.

…atch error measurement. updated jupyter script to use filedialog to select csv. reverted the mysterious lowpass filter in the cart target position so that the position is changed instantaneously. added partially working M key command to cartpole_trajectory_generator.py to allow switching dance step from keyboard. added keyboard_input() and print_keyboard_help() to template_controller and implemented it in controller_mpc.py and cartpole_trajectory_generator.py. added easygui to requirements.txt. added warning to __init__.py in SI_Toolkit to warn about slow tensorflow eager mode. moved is_physical_cartpole_runnning_and_control_enabled to CartPole.__init__.

remove terminal cost in cartpole_dancer_cost.py to move track edge barrier cost inside stage costs function where it is easier to manage. comment out terminal cost computation in Control_Toolkit - note this will break cost functions that use it.

commented out some noisy log.debug. made horizon same as physical cartpole

updated satisfaction CSV to use markers from audition and added more cartsheels

fixed import of CompileTF in optimizer_rpgd_tf.py. other small code changes and comments

… that it is shown in live console display instead of scrolling away. fixed cartonly trajectory so that cartonly works with sawtooth much better. changed PhysicalCartPoleDriver.py logger so that the handler logs with \r like other status indicators. added another exception handler in globals_and_utils.py to handle AttributeErrors in config files. Optimized shimmy amplitudes in satisfaction

…name in console logger for logging during control

…e handler once

…or for compatibilty with RPGD gradient computation. update_attributes in globals_and_utils.py supports assignment of tf.Variable and now raises exception if we try to .assign an immutable Tensor. Balance step now uses int +1 or -1 for desired pole up or down, not 'up' or 'down'. computation_library.py now casts variale to numpy() from tensorflow only if it is a Tensor type.

…pole balancing

marcinpaluch1994 and others added 12 commits November 24, 2022 16:17

Starting template for integrating vector of target positions

68bf537

docstring for update_parameters

d1d9203

controller_mpc.py now copies all modified controller, cost and optimi…

cc2b58c

…zer config changes to updated_attributes before executing optimizer.step

got basic shimmy movement to work now. added helper vars to access co…

fef38b0

…nfig values in cartpole_trajectory_generator.py. added dt to params in cartpole_trajectory_generator.step() call.

added cartonly trajectory and fixed bug that erased the target positi…

d2e9942

…on and equilibrium from updated attributes before they were updated in controller

added MPPI papers to docstring

ab0f34e

added MPPI papers to docstring

7ad7be1

frehe commented Dec 14, 2022

View reviewed changes

requirements.txt

gym

Copy link

Member Author

frehe Dec 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

gym

gymnasium

Package name changed

frehe and others added 12 commits December 14, 2022 15:46

Rename num_rollouts -> batch_size in config templates

86c8d68

add comment

3ade5d0

local changes, all minor except for trajectory cost that is in flux

56d3e51

Merge remote-tracking branch 'origin/Tobi_Dance' into Tobi_Dance

9dad3a7

added dancer that reads CSV file to specify sequence of 'steps' (beha…

1f324f4

…viors).

fixed import of CompileTF to point to SI_Toolkit

0b543a0

merged from Tobi_Dance and added some loggers

c0ee620

moved get_logger to own file in SI_Toolkit

ee9ca38

Started merging from remote/origin/Tobi_Dance

Merge remote-tracking branch 'origin/Tobi_Dance' into Tobi_Dance

ecf2cc8

# Conflicts: # Controllers/__init__.py # Controllers/controller_mpc.py # Cost_Functions/cost_function_wrapper.py # Optimizers/optimizer_rpgd_tf.py # others/environment.py # others/globals_and_utils.py

added search path for running from physical-cartpole.

69b886c

fixed logger->log renaming

tobidelbruck added 19 commits February 7, 2023 08:51

update path to config_cost_functions.yml

4ce131e

move get_logger.py to Control_Toolkit so that it can be used by physi…

32c0a50

…cal-cartpole repo. Fix some cartpole parameter import. Fix rename of config_cost_functions.yml. Add timer.py and yes_or_no.py utility classes.

Merge remote-tracking branch 'origin/Tobi_Dance' into Tobi_Dance

45f1d56

added some docstrings, but they are not very informative

e2162fd

setting up to measure model mismatch in controller_mpc.py, not implemented yet.

added 'cartwheel' step to cartpole_trajectory_generator.py.

6b8a676

commented out some noisy log.debug. made horizon same as physical cartpole

added warning for cart calibration.

039d385

fixed some logic and reduced some loggers to debug level

ed2c118

updated satisfaction CSV to use markers from audition and added more cartsheels

fixed shimmy math.

a16b454

fixed import of CompileTF in optimizer_rpgd_tf.py. other small code changes and comments

Merge remote-tracking branch 'origin/Tobi_Dance' into Tobi_Dance

000ff67

improved logging output to make debug logger light gray, include file…

6e8ef51

…name in console logger for logging during control

fixed get_logger.py that now uses a single logger name to only add th…

e894dfc

…e handler once

reduced chatter in logging

3489a44

initial commit of Shreyan's code for energy-based controller for cart…

807c46d

…pole balancing

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Draft: Tobi Dance Branch #12

Draft: Tobi Dance Branch #12

frehe commented Dec 14, 2022

frehe Dec 14, 2022

frehe Dec 14, 2022

frehe Dec 14, 2022

frehe Dec 14, 2022

frehe Dec 14, 2022 •

edited

Loading

frehe Dec 14, 2022

	def __init__(self, controller: template_controller, ComputationLib: "type[ComputationLibrary]", config:dict=None) -> None:
	def __init__(self, controller: template_controller, ComputationLib: "type[ComputationLibrary]", config:"Optional[dict]"=None) -> None:

	self, state_horizon: TensorType, inputs: TensorType, previous_input: TensorType = None, config:dict=None
	self, state_horizon: TensorType, inputs: TensorType, previous_input: TensorType = None, config:"Optional[dict]"=None

	from types import MappingProxyType
	from types import MappingProxyType
	from typing import Optional

Draft: Tobi Dance Branch #12

Are you sure you want to change the base?

Draft: Tobi Dance Branch #12

Conversation

frehe commented Dec 14, 2022

frehe Dec 14, 2022

Choose a reason for hiding this comment

frehe Dec 14, 2022

Choose a reason for hiding this comment

frehe Dec 14, 2022

Choose a reason for hiding this comment

frehe Dec 14, 2022

Choose a reason for hiding this comment

frehe Dec 14, 2022 • edited Loading

Choose a reason for hiding this comment

frehe Dec 14, 2022

Choose a reason for hiding this comment

frehe Dec 14, 2022 •

edited

Loading