Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Polish CodeQL and do some chores #2255

Merged
merged 10 commits into from
Feb 22, 2024
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/benchmarks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,11 @@ jobs:
sudo apt-get update
sudo apt-get install -y ninja-build ccache
- name: Set up cmake
uses: jwlawson/actions-setup-cmake@v1.13
uses: jwlawson/actions-setup-cmake@v2.0.0
with:
cmake-version: 3.22.x
- name: Clone w/ submodules
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
submodules: recursive
# figure out vector extensions for ccache key
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/build-pip-wheels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:

steps:
- name: Get Arbor
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
submodules: recursive
- name: Create unique version in pyproject.toml
Expand All @@ -38,7 +38,7 @@ jobs:

steps:
- name: Get Arbor
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
submodules: recursive
- name: Delete pyproject.toml
Expand Down Expand Up @@ -69,7 +69,7 @@ jobs:
- name: Get packages
run: python3 -m pip install build
- name: Get Arbor
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
submodules: recursive
- name: Delete pyproject.toml
Expand Down Expand Up @@ -118,7 +118,7 @@ jobs:
needs: [build_binary_wheels, build_sdist]
steps:
- name: "Clone w/ submodules"
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
fetch-depth: 0
submodules: "recursive"
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,13 +46,13 @@ jobs:

steps:
- name: Checkout repository
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
submodules: recursive

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
Expand Down Expand Up @@ -81,6 +81,6 @@ jobs:
# ./location_of_script_within_repo/buildscript.sh

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
uses: github/codeql-action/analyze@v3
with:
category: "/language:${{matrix.language}}"
6 changes: 3 additions & 3 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,17 @@ jobs:
strategy:
fail-fast: false
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: Get packages
run: |
python -m pip install --upgrade pip
pip install flake8
- name: Clone w/ submodules
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
submodules: recursive
- name: Python Formatting
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/sanitize.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,11 @@ jobs:
sudo apt-get update
sudo apt-get install -y ninja-build ccache
- name: Set up cmake
uses: jwlawson/actions-setup-cmake@v1.13
uses: jwlawson/actions-setup-cmake@v2.0.0
with:
cmake-version: 3.19.x
- name: Clone w/ submodules
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
submodules: recursive
- name: Update pip
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/test-docs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,17 +18,17 @@ jobs:
python-version: [3.8]
steps:
- name: Set up cmake
uses: jwlawson/actions-setup-cmake@v1.13
uses: jwlawson/actions-setup-cmake@v2.0.0
with:
cmake-version: ${{ matrix.config.cmake }}
- name: Set up Python
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Update pip and setup venv
run: python -m pip install --upgrade pip && python -m venv ~/env && . ~/env/bin/activate && echo "PATH=$PATH" >> $GITHUB_ENV
- name: Clone w/ submodules
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
submodules: recursive
- name: Install Python packages
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/test-matrix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -105,19 +105,19 @@ jobs:
run: |
brew install openmpi ninja ccache
- name: Set up cmake
uses: jwlawson/actions-setup-cmake@v1.13
uses: jwlawson/actions-setup-cmake@v2.0.0
with:
cmake-version: ${{ matrix.config.cmake }}
- name: Set up Python
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.config.py }}
- name: Update pip
run: python -m pip install --upgrade pip
- name: Install Python packages
run: pip install numpy sphinx svgwrite sphinx-rtd-theme mpi4py pandas seaborn
- name: Clone w/ submodules
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
submodules: recursive
# figure out vector extensions for ccache key
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/test-pip.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,11 @@ jobs:
os: [ubuntu-20.04]
python-version: [3.8]
steps:
- uses: actions/setup-python@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Clone w/ submodules
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
submodules: recursive
- name: Update pip and setup venv
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test-spack.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jobs:
sudo apt-get install -y gfortran libblas-dev

- name: clone arbor
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
path: arbor
submodules: recursive
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
[![gitpod](https://img.shields.io/badge/Gitpod-Ready--to--Code-blue?logo=gitpod)](https://gitpod.io/#https://github.com/arbor-sim/arbor)
[![docs](https://readthedocs.org/projects/arbor/badge/?version=latest)](https://docs.arbor-sim.org/en/latest/)
[![gitter](https://badges.gitter.im/arbor-sim/community.svg)](https://gitter.im/arbor-sim/community)
[![CodeQL](https://github.com/arbor-sim/arbor/actions/workflows/codeql.yml/badge.svg?branch=master)](https://github.com/arbor-sim/arbor/actions/workflows/codeql.yml)

# Arbor Library

Expand Down
25 changes: 9 additions & 16 deletions arbor/cable_cell_group.cpp
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
#include <functional>
#include <optional>
#include <unordered_set>
#include <variant>
#include <vector>

Expand All @@ -18,8 +15,6 @@
#include "cable_cell_group.hpp"
#include "profile/profiler_macro.hpp"
#include "sampler_map.hpp"
#include "util/filter.hpp"
#include "util/maputil.hpp"
#include "util/partition.hpp"
#include "util/range.hpp"
#include "util/span.hpp"
Expand Down Expand Up @@ -215,9 +210,9 @@ void run_samples(
fvm_probe_scratch& scratch)
{
const sample_size_type n_raw_per_sample = p.raw_handles.size();
sample_size_type n_sample = (sc.end_offset-sc.begin_offset)/n_raw_per_sample;
arb_assert((sc.end_offset-sc.begin_offset)==n_sample*n_raw_per_sample);
arb_assert((unsigned)n_raw_per_sample==p.weight.size());
sample_size_type n_sample = (sc.end_offset - sc.begin_offset)/n_raw_per_sample;
arb_assert((sc.end_offset - sc.begin_offset)==n_sample*n_raw_per_sample);
arb_assert((unsigned)n_raw_per_sample == p.weight.size());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these changes to make CodeQL happy, or just for readability?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Readability. I can get behind a*b + c for similarities with mathematical notation, but =, ==, and especially - tend to merge visually into one token. Or: Too similar to snake_case for my liking.
As much of the code, especially newer parts is in this style, I tend to adjust these expression in passing when editing files. Same for removing redundant headers, which my IDE picks up on these days.


auto& sample_ranges = std::get<std::vector<cable_sample_range>>(scratch);
sample_ranges.clear();
Expand Down Expand Up @@ -249,14 +244,12 @@ void run_samples(
sc.sampler({sc.probeset_id, sc.index, p.get_metadata_ptr()}, n_sample, sample_records.data());
}

void run_samples(
const fvm_probe_interpolated_multi& p,
const sampler_call_info& sc,
const arb_value_type* raw_times,
const arb_value_type* raw_samples,
std::vector<sample_record>& sample_records,
fvm_probe_scratch& scratch)
{
void run_samples(const fvm_probe_interpolated_multi& p,
const sampler_call_info& sc,
const arb_value_type* raw_times,
const arb_value_type* raw_samples,
std::vector<sample_record>& sample_records,
fvm_probe_scratch& scratch) {
const sample_size_type n_raw_per_sample = p.raw_handles.size();
const sample_size_type n_interp_per_sample = n_raw_per_sample/2;
sample_size_type n_sample = (sc.end_offset-sc.begin_offset)/n_raw_per_sample;
Expand Down
17 changes: 7 additions & 10 deletions arbor/communication/dry_run_context.cpp
Original file line number Diff line number Diff line change
@@ -1,12 +1,10 @@
#include <algorithm>
#include <string>
#include <vector>

#include <arbor/spike.hpp>

#include "distributed_context.hpp"
#include "label_resolution.hpp"
#include "threading/threading.hpp"
#include "util/rangeutil.hpp"

namespace arb {
Expand All @@ -26,7 +24,7 @@ struct dry_run_context_impl {
count_type local_size = local_spikes.size();

std::vector<spike> gathered_spikes;
gathered_spikes.reserve(local_size*num_ranks_);
gathered_spikes.reserve(std::size_t{local_size}*num_ranks_);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't change the (effective) maximum size of the gathered_spikes vector without also changing the range of the indices in the loop 35-37.

We ultimately make a partition of the indices into the variable partition which has count_type as its value type, and this vector is used to construct the gathered_vector return type, which also expects things in terms of count_type. I believe the correct fix is not to cast local_size or use std::size_t here but instead

  1. Add an assertion which confirms that the local_size*num_ranks_ doesn't wrap in count_type, and
  2. Uses std::size_t{local_size*num_ranks_} to keep CodeQL happy or else we tell CodeQL that this is not a bug (my preference).


for (count_type i = 0; i < num_ranks_; i++) {
util::append(gathered_spikes, local_spikes);
Expand All @@ -49,25 +47,24 @@ struct dry_run_context_impl {
void remote_ctrl_send_done() const {}
gathered_vector<cell_gid_type>
gather_gids(const std::vector<cell_gid_type>& local_gids) const {
using count_type = typename gathered_vector<cell_gid_type>::count_type;

count_type local_size = local_gids.size();
size_t local_size = local_gids.size();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

std::size_t?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes and all other instance in that file.


std::vector<cell_gid_type> gathered_gids;
gathered_gids.reserve(local_size*num_ranks_);

for (count_type i = 0; i < num_ranks_; i++) {
for (size_t i = 0; i < num_ranks_; i++) {
util::append(gathered_gids, local_gids);
}

for (count_type i = 0; i < num_ranks_; i++) {
for (count_type j = i*local_size; j < (i+1)*local_size; j++){
for (size_t i = 0; i < num_ranks_; i++) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 more std::size_ts

for (size_t j = i*local_size; j < (i+1)*local_size; j++){
gathered_gids[j] += num_cells_per_tile_*i;
}
}

using count_type = typename gathered_vector<cell_gid_type>::count_type;
std::vector<count_type> partition;
for (count_type i = 0; i <= num_ranks_; i++) {
for (size_t i = 0; i <= num_ranks_; i++) {
partition.push_back(static_cast<count_type>(i*local_size));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no point doing everything in size_t only to cast back down to count_type here; the alterations just make the potential bug more obscure. IMO the only correct resolution is to have an assert that checks that the product doesn't wrap and then to keep using count_type.

}

Expand Down
4 changes: 2 additions & 2 deletions arbor/include/arbor/common_types.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -138,8 +138,8 @@ using time_type = double;
constexpr time_type terminal_time = std::numeric_limits<time_type>::max();

// For holding counts and indexes into generated sample data.

using sample_size_type = std::int32_t;
using sample_index_type = std::int32_t;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where is this type actually used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It -- currently -- isn't, but it's useful when we are considering things like:

sample_size_type a = ..., b = ..., c = a - b;

due to the potential overflow.

using sample_size_type = std::uint32_t;

// Enumeration for execution back-end targets, as specified in domain decompositions.

Expand Down
3 changes: 2 additions & 1 deletion arbor/include/arbor/math.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,8 @@ template <
>
C round_up(T v, U b) {
C m = v%b;
return v-m+signum(m)*impl::abs_if_signed(b, Signed{});
C sign = signum(m);
return v - m + sign*impl::abs_if_signed(b, Signed{});
}

// Returns 1/x if x != 0; 0 otherwise
Expand Down
5 changes: 3 additions & 2 deletions arbor/include/arbor/util/hash_def.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,9 @@ namespace detail {
template <typename T>
inline constexpr std::size_t internal_hash(T&& data) {
using D = std::decay_t<T>;
constexpr std::size_t prime = 0x100000001b3;
constexpr std::size_t offset_basis = 0xcbf29ce484222325;
// Due to constexpr if we might never use these.
[[maybe_unused]] constexpr std::size_t prime = 0x100000001b3;
[[maybe_unused]] constexpr std::size_t offset_basis = 0xcbf29ce484222325;
static_assert(!std::is_pointer_v<D> || std::is_same_v<D, void*> || std::is_convertible_v<T, std::string_view>,
"Pointer types except void* will not be hashed.");
if constexpr (std::is_convertible_v<T, std::string_view>) {
Expand Down
2 changes: 0 additions & 2 deletions example/generators/generators.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -82,8 +82,6 @@ class generator_recipe: public arb::recipe {
std::vector<arb::event_generator> event_generators(cell_gid_type gid) const override {
assert(gid==0); // There is only one cell in the model

using RNG = std::mt19937_64;

auto hz_to_freq = [](double hz) { return hz*1e-3; };
time_type t0 = 0;

Expand Down
12 changes: 7 additions & 5 deletions modcc/io/prefixbuf.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -75,11 +75,13 @@ int indent_manip::xindex() {

static void apply_indent_prefix(std::ios& s, int index) {
if (auto pbuf = dynamic_cast<prefixbuf*>(s.rdbuf())) {
indent_stack* stack_ptr = static_cast<indent_stack*>(s.pword(index));
unsigned tabwidth = s.iword(index);

unsigned tabs = (!stack_ptr || stack_ptr->empty())? 0: stack_ptr->top();
pbuf->prefix = std::string(tabs*tabwidth, ' ');
if (auto ptr = static_cast<indent_stack*>(s.pword(index)); ptr && !ptr->empty()) {
unsigned n_tab = ptr->top();
Copy link
Contributor

@halfflat halfflat Feb 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Argh! This is terribly misleading formatting.
Edit: what was so bad with unsigned tabs = (!stack_ptr || stack_ptr->empty())? 0: stack_ptr->top();?

Copy link
Contributor Author

@thorstenhater thorstenhater Feb 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I knew you were going to love it. Isn't it beautiful how the second line of the condition and the first line of the body line up?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It has a certain horrible charm to it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed it.

unsigned width = s.iword(index);
pbuf->prefix = std::string(n_tab*width, ' ');

Check failure

Code scanning / CodeQL

Multiplication result converted to larger type High

Multiplication result may overflow 'unsigned int' before it is converted to 'size_type'.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dismissed.

} else {
pbuf->prefix = std::string();
}
}
}

Expand Down
Loading