Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RUBY-2523 Client Side Operations Timeout #2882

Merged
merged 39 commits into from
Aug 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
e1158d3
RUBY-3370 Add CSOT to server selection and connection checkout (#2833)
comandeo-mongo Feb 14, 2024
955d4f5
RUBY-3368 Measure minimum RTT (#2837)
comandeo-mongo Feb 14, 2024
6817971
RUBY-3368 Set maxTimeMS (#2840)
comandeo-mongo Feb 20, 2024
752c1f4
Add write timeout (#2843)
comandeo-mongo Mar 6, 2024
1af575a
Merge remote-tracking branch 'upstream/master' into csot
comandeo-mongo Mar 6, 2024
8ddd727
Merge remote-tracking branch 'upstream/master' into csot
comandeo-mongo Mar 7, 2024
e915e84
RUBY-3372 Add CSOT for CRUD commands (#2849)
comandeo-mongo Mar 10, 2024
0b7c157
Merge remote-tracking branch 'upstream/master' into csot
comandeo-mongo Mar 20, 2024
8961391
3376 csot for sessions (#2850)
comandeo-mongo Mar 21, 2024
d5df239
Fix csot test failures (#2853)
comandeo-mongo Apr 2, 2024
9fd0999
Merge branch 'master' into csot
comandeo-mongo Apr 2, 2024
2b9f86f
RUBY-3373 CSOT for retries (#2857)
comandeo-mongo Apr 9, 2024
dd67a60
Merge branch 'master' into csot
comandeo-mongo Apr 10, 2024
4133a25
RUBY-3379 CSOT for bulk writes (#2859)
comandeo-mongo Apr 12, 2024
76869d4
RUBY-3444 Server timeout error (#2860)
jamis Apr 18, 2024
d42b312
RUBY-3445 Add timeouts in OpMsg instead of Protocol::Message (#2861)
jamis Apr 18, 2024
ba6c83b
RUBY-3390 Add CSOT to list_databases (#2862)
comandeo-mongo Apr 19, 2024
66c6c2f
RUBY-3372 CSOT Cursors (#2863)
jamis Apr 22, 2024
d4974a0
RUBY-3374 CSOT change streams (#2865)
jamis Apr 24, 2024
acf370b
RUBY-3390 CSOT for lisCollections (#2866)
comandeo-mongo Apr 24, 2024
8625ba5
Merge branch 'master' into csot
comandeo-mongo Apr 25, 2024
d0fe3fb
Fix failing specs in CSOT branch (#2867)
jamis May 6, 2024
bba73e9
RUBY-3375 CSOT for CSFLE (#2868)
comandeo-mongo May 13, 2024
aceef6e
RUBY-3379 CSOT for bulk writes (#2871)
comandeo-mongo May 16, 2024
735daf6
Sync CSOT spec tests (#2872)
comandeo-mongo May 23, 2024
ab7782c
Merge branch 'master' into csot
comandeo-mongo Jun 5, 2024
c75533a
Merge remote-tracking branch 'upstream/master' into csot
comandeo-mongo Jun 17, 2024
06e8a38
Merge branch 'master' into csot
comandeo-mongo Jul 1, 2024
4468400
csot
comandeo-mongo Jul 1, 2024
dbb8ce3
Disable flaky tests
comandeo-mongo Jul 1, 2024
2cad173
Add synchronization
comandeo-mongo Jul 1, 2024
5169802
Retry more flaky tests
comandeo-mongo Jul 2, 2024
4617dda
Fix server spec
comandeo-mongo Jul 2, 2024
e487cb5
Fix change streams spec
comandeo-mongo Jul 2, 2024
03b3613
Fix change streams spec
comandeo-mongo Jul 2, 2024
b4d8a17
Merge branch 'master' into csot
comandeo-mongo Jul 4, 2024
0c365c7
RUBY-3378 Document CSOT (#2884)
comandeo-mongo Jul 9, 2024
e5fdcf4
Merge branch 'master' into csot
comandeo-mongo Jul 19, 2024
076cf30
RUBY-3407 Add spec test (#2890)
comandeo-mongo Jul 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .evergreen/aws_lambda
45 changes: 40 additions & 5 deletions .evergreen/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,25 @@ functions:

EOT

"run CSOT tests":
- command: shell.exec
type: test
params:
shell: bash
working_dir: "src"
script: |
${PREPARE_SHELL}
# Needed for generating temporary aws credentials.
if [ -n "${FLE}" ];
then
export AWS_ACCESS_KEY_ID="${fle_aws_key}"
export AWS_SECRET_ACCESS_KEY="${fle_aws_secret}"
export AWS_DEFAULT_REGION="${fle_aws_region}"
fi
export CSOT_SPEC_TESTS=1
TEST_CMD="bundle exec rspec spec/spec_tests/client_side_operations_timeout_spec.rb" \
.evergreen/run-tests.sh

"export FLE credentials":
- command: shell.exec
type: test
Expand Down Expand Up @@ -748,6 +767,9 @@ tasks:
- name: "test-kerberos"
commands:
- func: "run Kerberos unit tests"
- name: "test-csot"
commands:
- func: "run CSOT tests"
- name: "test-fle"
commands:
- func: "export FLE credentials"
Expand Down Expand Up @@ -863,6 +885,7 @@ tasks:
AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
AWS_SESSION_TOKEN: ${AWS_SESSION_TOKEN}
LAMBDA_STACK_NAME: "dbx-ruby-lambda"
CLUSTER_PREFIX: "dbx-ruby-lambda"
RVM_RUBY: ruby-3.2
MONGODB_URI: ${MONGODB_URI}
axes:
Expand Down Expand Up @@ -941,6 +964,10 @@ axes:
display_name: Replica Set
variables:
TOPOLOGY: replica-set
- id: "replica-set-single-node"
display_name: Replica Set (Single Node)
variables:
TOPOLOGY: replica-set-single-node
- id: "sharded-cluster"
display_name: Sharded
variables:
Expand Down Expand Up @@ -1373,6 +1400,16 @@ buildvariants:
tasks:
- name: "test-mlaunch"

- matrix_name: CSOT
matrix_spec:
ruby: "ruby-3.2"
mongodb-version: "7.0"
topology: replica-set-single-node
os: rhel8
display_name: "CSOT - ${mongodb-version}"
tasks:
- name: test-csot

- matrix_name: "no-retry-reads"
matrix_spec:
retry-reads: no-retry-reads
Expand Down Expand Up @@ -1411,7 +1448,7 @@ buildvariants:
lint: on
ruby: "ruby-3.2"
mongodb-version: "7.0"
topology: '*'
topology: ["standalone", "replica-set", "sharded-cluster"]
os: rhel8
display_name: "${mongodb-version} ${topology} ${lint} ${ruby}"
tasks:
Expand Down Expand Up @@ -1840,11 +1877,9 @@ buildvariants:

- matrix_name: "serverless"
matrix_spec:
# https://jira.mongodb.org/browse/RUBY-3217
# ruby: ["ruby-3.2", "ruby-3.1", "ruby-3.0", "ruby-2.7", "jruby-9.4", "jruby-9.3"]
ruby: ["ruby-3.2", "ruby-3.1", "ruby-3.0", "ruby-2.7"]
ruby: "ruby-3.2"
fle: path
os: rhel8
os: ubuntu2204
display_name: "Atlas serverless ${ruby}"
tasks:
- name: serverless_task_group
Expand Down
4 changes: 4 additions & 0 deletions .evergreen/config/axes.yml.erb
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,10 @@ axes:
display_name: Replica Set
variables:
TOPOLOGY: replica-set
- id: "replica-set-single-node"
display_name: Replica Set (Single Node)
variables:
TOPOLOGY: replica-set-single-node
- id: "sharded-cluster"
display_name: Sharded
variables:
Expand Down
23 changes: 23 additions & 0 deletions .evergreen/config/common.yml.erb
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,25 @@ functions:

EOT

"run CSOT tests":
- command: shell.exec
type: test
params:
shell: bash
working_dir: "src"
script: |
${PREPARE_SHELL}
# Needed for generating temporary aws credentials.
if [ -n "${FLE}" ];
then
export AWS_ACCESS_KEY_ID="${fle_aws_key}"
export AWS_SECRET_ACCESS_KEY="${fle_aws_secret}"
export AWS_DEFAULT_REGION="${fle_aws_region}"
fi
export CSOT_SPEC_TESTS=1
TEST_CMD="bundle exec rspec spec/spec_tests/client_side_operations_timeout_spec.rb" \
.evergreen/run-tests.sh

"export FLE credentials":
- command: shell.exec
type: test
Expand Down Expand Up @@ -745,6 +764,9 @@ tasks:
- name: "test-kerberos"
commands:
- func: "run Kerberos unit tests"
- name: "test-csot"
commands:
- func: "run CSOT tests"
- name: "test-fle"
commands:
- func: "export FLE credentials"
Expand Down Expand Up @@ -860,5 +882,6 @@ tasks:
AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
AWS_SESSION_TOKEN: ${AWS_SESSION_TOKEN}
LAMBDA_STACK_NAME: "dbx-ruby-lambda"
CLUSTER_PREFIX: "dbx-ruby-lambda"
RVM_RUBY: ruby-3.2
MONGODB_URI: ${MONGODB_URI}
20 changes: 15 additions & 5 deletions .evergreen/config/standard.yml.erb
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@

recent_mdb = %w( 6.0 5.3 )
latest_5x_mdb = "5.3".inspect # so it gets quoted as a string

all_dbs = %w(latest 7.0 6.0 5.3 5.0 4.4 4.2 4.0 3.6)
%>

buildvariants:
Expand Down Expand Up @@ -142,6 +144,16 @@ buildvariants:
tasks:
- name: "test-mlaunch"

- matrix_name: CSOT
matrix_spec:
ruby: <%= latest_ruby %>
mongodb-version: <%= latest_stable_mdb %>
topology: replica-set-single-node
os: rhel8
display_name: "CSOT - ${mongodb-version}"
tasks:
- name: test-csot

- matrix_name: "no-retry-reads"
matrix_spec:
retry-reads: no-retry-reads
Expand Down Expand Up @@ -180,7 +192,7 @@ buildvariants:
lint: on
ruby: <%= latest_ruby %>
mongodb-version: <%= latest_stable_mdb %>
topology: '*'
topology: <%= topologies %>
os: rhel8
display_name: "${mongodb-version} ${topology} ${lint} ${ruby}"
tasks:
Expand Down Expand Up @@ -512,11 +524,9 @@ buildvariants:

- matrix_name: "serverless"
matrix_spec:
# https://jira.mongodb.org/browse/RUBY-3217
# ruby: <%= supported_rubies %>
ruby: <%= supported_mri_rubies %>
ruby: <%= latest_ruby %>
fle: path
os: rhel8
os: ubuntu2204
display_name: "Atlas serverless ${ruby}"
tasks:
- name: serverless_task_group
Expand Down
1 change: 1 addition & 0 deletions .evergreen/handle-paths.sh
118 changes: 0 additions & 118 deletions .evergreen/run-deployed-lambda-aws-tests.sh

This file was deleted.

2 changes: 1 addition & 1 deletion .evergreen/run-tests-deployed-lambda.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,4 @@ export MONGODB_URI=${MONGODB_URI}
export CLUSTER_PREFIX="ruby-driver-"
export TEST_LAMBDA_DIRECTORY=`dirname "$0"`/../spec/faas/ruby-sam-app

. `dirname "$0"`/run-deployed-lambda-aws-tests.sh
. `dirname "$0"`/aws_lambda/run-deployed-lambda-aws-tests.sh
10 changes: 5 additions & 5 deletions .evergreen/run-tests-serverless.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,16 +25,16 @@ else
python3 -u .evergreen/mongodl.py --component crypt_shared -V ${SERVERLESS_MONGODB_VERSION} --out `pwd`/csfle_lib --target `host_distro` || true
if test -f `pwd`/csfle_lib/lib/mongo_crypt_v1.so
then
echo Usinn crypt shared library version ${SERVERLESS_MONGODB_VERSION}
echo Using crypt shared library version ${SERVERLESS_MONGODB_VERSION}
export MONGO_RUBY_DRIVER_CRYPT_SHARED_LIB_PATH=`pwd`/csfle_lib/lib/mongo_crypt_v1.so
else
echo Failed to download crypt shared library
exit -1
fi
fi

if ! ( test -f /etc/os-release & grep -q ^ID.*rhel /etc/os-release & grep -q ^VERSION_ID.*8.0 /etc/os-release ); then
echo Serverless tests assume rhel80
if ! ( test -f /etc/os-release & grep -q ^ID.*ubuntu /etc/os-release & grep -q ^VERSION_ID.*22.04 /etc/os-release ); then
echo Serverless tests assume ubuntu2204
echo If this has changed, update .evergreen/run-tests-serverless.sh as necessary
exit -1
fi
Expand All @@ -43,8 +43,8 @@ mkdir libmongocrypt
cd libmongocrypt
curl --retry 3 -fLo libmongocrypt-all.tar.gz "https://s3.amazonaws.com/mciuploads/libmongocrypt/all/master/latest/libmongocrypt-all.tar.gz"
tar xf libmongocrypt-all.tar.gz
# We assume that serverless tests always use rhel80
export LIBMONGOCRYPT_PATH=`pwd`/rhel-80-64-bit/nocrypto/lib64/libmongocrypt.so
# We assume that serverless tests always use ubuntu2204
export LIBMONGOCRYPT_PATH=`pwd`/ubuntu2204-64/nocrypto/lib/libmongocrypt.so
cd -

cd .evergreen/csfle
Expand Down
6 changes: 5 additions & 1 deletion .evergreen/run-tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ fi

calculate_server_args
launch_ocsp_mock

launch_server "$dbdir"

uri_options="$URI_OPTIONS"
Expand All @@ -90,6 +91,9 @@ elif test "$TOPOLOGY" = replica-set; then
# or it can try to send the commands to secondaries.
hosts=localhost:27017,localhost:27018
uri_options="$uri_options&replicaSet=test-rs"
elif test "$TOPOLOGY" = replica-set-single-node; then
hosts=localhost:27017
uri_options="$uri_options&replicaSet=test-rs"
else
hosts=localhost:27017
fi
Expand Down Expand Up @@ -283,7 +287,7 @@ fi

set_fcv

if test "$TOPOLOGY" = replica-set && ! echo "$MONGODB_VERSION" |fgrep -q 2.6; then
if test "$TOPOLOGY" = replica-set || test "$TOPOLOGY" = replica-set-single-node; then
ruby -Ilib -I.evergreen/lib -rbundler/setup -rserver_setup -e ServerSetup.new.setup_tags
fi

Expand Down
2 changes: 1 addition & 1 deletion .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@
url = https://github.com/mongodb-labs/drivers-evergreen-tools
[submodule "spec/shared"]
path = spec/shared
url = git@github.com:mongodb-labs/mongo-ruby-spec-shared.git
url = git@github.com:mongodb-labs/mongo-ruby-spec-shared.git
Loading
Loading