Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

…g APIs (#2414)

* chore: add `google/shopping/type/types.proto` when generating shopping APIs.

* do not copy types.proto

* feat: add new-client.py

* refactor code

* clone googleapis

* add build parser

* parse inputs from BUILD

* find versioned dir within proto_path

* fix postprocessing, use google-java-format

* migirate shell scripts

* change depth

* add generate_gapic_bom.sh

* process root pom

* only generating gapic-libraries-bom if it already exists

* add consolidate_config.sh

* add generation dir

* use variable for script dir

* add set_parent_pom.sh

* add apply_current_versions.sh

* add click option for versions.txt

* change dir

* change find command

* add readme_update.sh

* change script path

* refactor

* code refactor

* add generator version

* use GitPython to checkout googleapis

* code refactor

* use versions.txt in the monorepo

* add comments

* change year

* change copyright year

* remove apply_current_versions.sh

* add __pychche__ to gitignore

* parse generator version from WORKSPACE

* remove readme_update.sh

* remove update_owlbot_postprocessor_config.sh

* remove delete_non_generated_samples.sh

* remove set_parent_pom.sh

* add destination_name

* restore format plugin

* remove gitignore

* remove consolidate_config.sh

* change output directory

---------

Co-authored-by: diegomarquezp <[email protected]>

…ies (#2426)

…ons (#2342)

* use local synthtool and owlbot

* remove unused files

* remove more unused files

* remove cache files in owlbot

* use java 11 for it

* remove kokoro files

* use glob in owlbot entrypoint

* remove unused files

* do not do post-process IT on mac

* concise entrypoint logic

* cleanup i

* cleanup ii

* cleanup iii

* cleanup iv

* remove templates

* remove protos folder

* remove synthtool

* connect image to owlbot entrypoint

* simplify synthtool docker run command

* install synthtool locally

* install synthtool only once

* use virtualenvs to run python scripts

* install pyenv in action

* remove jar from history

* download google-java-format

* fix pyenv init

* attempt to fix pyenv installation in gh action

* fix manual pyenv installation

* install pyenv in profile

* install pyenv in bashrc as well

* use bash shell explicitly in gh action

* install pyenv in same step as IT

* do not restart shell

* set pyenv path manually

* install pyenv in its own step

* propagate environment variables to other steps

* fix global env var setup

* remove wrong env settings

* explain usage of pyenv in README

* simplify pyenv setup

* add comment to owlbot entrypoint

* rename destination_path to preprocessed_libraries_path

* infer scripts_root in postprocess_library.sh

* use temporary folder for preprocess step

* use owlbot files from workspace

* get rid of output_folder argument

* use common temp dir to clone synthtool into

* lock synthtool to a specific commitish

* fix file transfer

* fix owl-bot-staging unpacking

* remove unnecessary workspace variable

* rename workspace to postprocessing_target

* remove owlbot sha logic

* remove repository_root variable

* cleanup

* correct pyenv comment

* clean temp sources folder on each run

* safety checks for get_proto_path_from_preprocessed_sources

* fix integration test

* disable compute and asset/v1p2beta1 temporarily

they have changes in googleapis that have not been reflected yet in
google-cloud-java

* fix unit tests

* correct comment

* do not install docker for macos

* fix owlbot files check

* fix license headers

* remove unnecessary owlbot_sha

* add explanation on why are there no macos + postprocess ITs

* use `fmt:format` instead of google formatter

* clean templates

* remove more unnecessary elements

* add README entry explaining owlbot maintenance

* remove unnecessary java format version

* wip: create generate_composed_library.sh

* initial implementation of generate_composed_library.sh

* partial implementation of scripts

* wip: fixes after main changes

* partial implementation of scripts ii

* correct arg parsing

* fixes in python utils

* fix generate_library call

* correct argument preparation

* add gapic generatior version arg check

* call generate_library successfully

* fix postprocessing step in generate_composed

* IT working for all libraries

* add unit tests

* fix comments in generate_composed_lib

* remove commented code

* prepare tests without postprocessing

* restore test functions

* fix rename utility for building owlbot folder

* correct linter problems

* install realpath on macos

* install utils also in unit tests

* include utilities.sh in showcase scripts

* comment py_util

* Update library_generation/generate_composed_library.sh

Co-authored-by: Tomo Suzuki <[email protected]>

* add comment explaining usage of coreutils in macos workflow

* explain that entrypoint.sh can be used for HW libraries

* use real world example for generate_composed_library.sh

* improve versions.txt explanation in generate_composed_library

* add return type in python utils

* fix versions file inference

* use ggj 2.29 to fix ITs temporarily

* disable asset due to licence year mismatch

* improve commment in generate_composed_library

* restore latest generator

* remove wrong WORKSPACE comment

* improve composed_library input comment

* Update library_generation/utilities.py

Co-authored-by: Blake Li <[email protected]>

* remove postprocessing logic in generate_library

* add generated_composed_library.py

* use python script in IT

* iterative fixes

* ignore python cache

* iterative fixes ii

* Update library_generation/generate_composed_library.py

Co-authored-by: Joe Wang <[email protected]>

* Update library_generation/utilities.py

Co-authored-by: Joe Wang <[email protected]>

* Update library_generation/generate_composed_library.py

Co-authored-by: Tomo Suzuki <[email protected]>

* use underscores in configuration yaml

* initial model for gapic config yaml

* add requirements file

* introduce yaml parsing logic

* move parse logic to GenerationConfig

* adapt composed_library script

* fixes i

* set IT to dummy mode

* pass BUILD parse utils to production utils

* fixes ii - constructor calls and composed_library arguments

* fix destination path for partial generations

* adapt IT to process individual libraries

* use proper versions in configuration yaml

* add rest of the libraries to integration test

* add library_type to config yaml

* reference to parent directory in compare_poms

* handle script failures

* use library-specific googleapis_commitish

* fix protobuf version

* install python reqs in Github Action

* fix python version path logic

* add python unit tests

* remove obsolete py_util tests

* add python unit tests in workflow

* correct type hinting for Library

* fix comments

* set enable_postprocessing input to main.py to boolean

* add explanation on library_type

* remove old proto_path_list

* fix comments in IT

* remove WORKSPACE file usage

* add IT configuration yaml for java-bigtable

* fix config yaml for bigtable

* finish tests for HW bigtable

* install python in gh action, lint fix

* update ggj to 2.32.0

* fix showcase test

* remove commented line

* use owlbot cli sha from config yaml

* use python version from config yaml

* use synthtool commitish from config yaml

* add repository_path option

* make destination_path required

* add typing

* use python version from config yaml

* correct workflow indentation

* fix workflow syntax

* use repository_path when postprocessing

* correct runs-on in workflow

* use full path to repo in workflow

* add debug output in workflow

* decompose steps to compute python version

* checout code in workflow

* fix function name in workflow

* use full path to obtain python version from yaml

* use correct path in python version workflow

* use set-output to share python version

* fix set-output call

* fix output setting in workflow

* correct python version letter case

* remove pyenv setup

* fix repository_path

* fix speech proto_path

* do not wipe out google-cloud-java in IT

* ensure correct version of python compares the poms

* fix diff check

* add return type for GenerationCOnfig.from_yaml

* add missing python unit tests

* use default values for enable_postprocessing

* use is_monorepo var, constant for google-coud-java

* fixes for local run

* compute monorepo variable heuristically

* update generation configs

* remove python version

* rename Library to LibraryConfig

* rename GAPIC to GapicConfig

* remove quotes from grpc version

* move ClientInputs to model folder

* parse BUILD file using ClientInputs

* add unit tests for ClientInputs

* fix CLientInputs typo

* fix synthtool version

* disable compute test

* fix unit test

* fix compute test

* fix unit tests

* remove BUILD parsing shell utilities

* update monorepo special treatment comment

* fix typo in shebang

---------

Co-authored-by: Tomo Suzuki <[email protected]>
Co-authored-by: Blake Li <[email protected]>
Co-authored-by: Joe Wang <[email protected]>

…2429)

* fix: Endpoint resolution uses the user set endpoint

* chore: Rename to userSetEndpoint()

* chore: Empty commit

* chore: Fix showcase tests

* feat: Opentelemetry implementation

…st channel (#2430)

Fixes #2423 
The root cause of the issue is that `logDirectPathMisconfig()` is called in the builder of `InstantiatingGrpcChannelProvider`, which could be called multiple times before it is fully instantiated. We should only call `logDirectPathMisconfig()` right before `createChannel()` which creates the first channel. 

We can not move it to before `createSingleChannel()` because it is used for resizing channel regularly after a client is initialized, and we only want to log direct path misconfiguration once.

* feat: autopopulate fields in the request

* revert showcase golden changes

* add AbstractTransportServiceStubClassComposerTest

* add REST implementation

* add GrpcDirectCallableTest

* remove unnecessary null check

* add field info proto registry and more unit test cases

* add HttpJsonDirectCallableTest

* refactor AbstractTransportServiceSTubClassComposer

* add more unit tests for Field

* feat: refactor request mutator expression

* fix goldens

* add showcase test for autopopulation

* fix lint

* change assertion in showcase test

* refactor for sonarcloud

* sonarcloud fixes

* sonarcloud

* sonarcloud fix

* fix sonarcloud

* slight refactoring

* revert changes to directCallable and replace with retryable Callable

* overload retrying Callables method

* change license header format

* fix license header

* fix showcase lint

* add comment

* add showcase comment

* add CallableTest and httpjson Retrying test

* fix lint

* add RetryingCallable test and some refactoring

* refactor GrpcCallableFactory

* remove extraneous from HttpJsonDirectCallableTest

* remove FakeHttpJsonChannel

* revert changes to tests for extra param

* refactoring

* Update gapic-generator-java/src/test/java/com/google/api/generator/gapic/model/FieldTest.java

Co-authored-by: Blake Li <[email protected]>

* Update gax-java/gax/src/test/java/com/google/api/gax/rpc/RetryingCallableTest.java

Co-authored-by: Blake Li <[email protected]>

---------

Co-authored-by: Blake Li <[email protected]>

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>