Skip to content

Commit

Permalink
add strawman
Browse files Browse the repository at this point in the history
  • Loading branch information
atharvas committed Mar 7, 2024
1 parent d61de90 commit 8d61586
Show file tree
Hide file tree
Showing 16 changed files with 8,150 additions and 4,445 deletions.
39 changes: 33 additions & 6 deletions index.html
Original file line number Diff line number Diff line change
Expand Up @@ -402,32 +402,59 @@ <h3 class="title is-size-6-mobile is-size-4-tablet">Symbolic Labelling: Concaten
</p>
</div>
<br />
<h3 class="title is-size-6-mobile is-size-4-tablet">Symbolic Embedding</h3>
<h3 class="title is-size-6-mobile is-size-4-tablet">Symbolic Encoding</h3>
<div class="content is-size-7-mobile is-size-6-tablet has-text-left step">
<p>
The symbolic labelling process is repeated for each entity in the scene. The resultant symbolic
vectors are stacked to form a <em>symbolic embedding</em> of the scene.
vectors are stacked to form a <em>symbolic encoding</em> of the scene.
</p>
</div>
<br />
<h3 class="title is-size-6-mobile is-size-4-tablet">Strawman #1</h3>
<div class="content is-size-7-mobile is-size-6-tablet has-text-left step">
<p>
Following <a href="https://arxiv.org/abs/2103.01937">NPS</a>, we will break down the transition function into two parts: Learning to select a module and learning to specialize the module to a task. However, how should we employ the neural and symbolic encodings?
</p>
<p>
The <strong style="color: #fb8c00;">symbolic encoding</strong> will help the selection module be robust to attribute compositions. However, if we just use the symbolic encoding, we will risk bottlenecking the model's ability to learn fine-grained dynamics-relevant attributes that may not be known ahead of time.
</p>
</div>
<br />
<h3 class="title is-size-6-mobile is-size-4-tablet">Strawman #2</h3>
<div class="content is-size-7-mobile is-size-6-tablet has-text-left step">
<p>
The <strong style="color: #417be4;">neural encoding</strong>, on the other hand, captures rich dynamics-relevant attributes which will enable good reconstruction. However, we will risk overfitting to attribute compositions seen during training.
</p>
</div>
<br />
<h3 class="title is-size-6-mobile is-size-4-tablet">Motivating a Neurosymbolic Encoding</h3>
<div class="content is-size-7-mobile is-size-6-tablet has-text-left step">
<p>
We solve this problem by employing a hybrid approach. We'll use the symbolic encoding to select a module and the neural encoding to predict the next state.
</p>
<p>
We'll now describe the rest of the architecture.
</p>
</div>
<br />
<h3 class="title is-size-6-mobile is-size-4-tablet">Module Selection</h3>
<div class="content is-size-7-mobile is-size-6-tablet has-text-left step">
<p>
The symbolic embedding is concatenated with the action vector after the embedding and action are
The symbolic encoding is concatenated with the action vector after the encoding and action are
reordered to match the canonical ordering of the symbols. The concatenated vector is select a
learnable module which is used to predict the next state.
</p>
<p>
Note that the symbolic embedding is
only selecting the module, which leaves room for the neural embedding to learn fine-grained
Note that the symbolic encoding is
only selecting the module, which leaves room for the neural encoding to learn fine-grained
dynamics-relevant attributes that may not be known ahead of time.
</p>
</div>
<br />
<h3 class="title is-size-6-mobile is-size-4-tablet">Module Application</h3>
<div class="content is-size-7-mobile is-size-6-tablet has-text-left step">
<p>
The selected module is applied to the neural embedding to predict the next state.
The selected module is applied to the neural encoding to predict the next state.
</p>
</div>
<br />
Expand Down
2 changes: 1 addition & 1 deletion static/onepass-inference/1.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1,674 changes: 854 additions & 820 deletions static/onepass-inference/10.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1,652 changes: 726 additions & 926 deletions static/onepass-inference/11.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1,446 changes: 493 additions & 953 deletions static/onepass-inference/12.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1,240 changes: 535 additions & 705 deletions static/onepass-inference/13.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1,992 changes: 1,992 additions & 0 deletions static/onepass-inference/14.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2,077 changes: 2,077 additions & 0 deletions static/onepass-inference/15.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
28 changes: 13 additions & 15 deletions static/onepass-inference/2.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
28 changes: 13 additions & 15 deletions static/onepass-inference/3.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
254 changes: 132 additions & 122 deletions static/onepass-inference/4.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
254 changes: 132 additions & 122 deletions static/onepass-inference/5.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
254 changes: 132 additions & 122 deletions static/onepass-inference/6.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
254 changes: 132 additions & 122 deletions static/onepass-inference/7.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
274 changes: 142 additions & 132 deletions static/onepass-inference/8.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1,127 changes: 743 additions & 384 deletions static/onepass-inference/9.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 8d61586

Please sign in to comment.