Inclusion Criteria: Inclusion Criteria Applicable to mCRPC Cohorts in Part 1 and PSMA-Targeted Radioligand-Naive mCRPC Cohort (Part 2 Cohort 2-2) * Have metastatic disease, defined by ≥ 1 metastatic lesion that is present on baseline computed tomography (CT), magnetic resonance imaging (MRI), or bone scan imaging * Have had prior bilateral orchiectomy and/or are receiving androgen deprivation therapy (ADT) and have a castrate level of serum testosterone (\< 50 ng/dL or \< 1.7 nmol/L) * Have documented progressive mCRPC based on ≥ 1 of the following criteria (per PCWG3): * PSA level ≥ 1 ng/mL that has increased on ≥ 2 successive occasions ≥ 1 week apart * Nodal or visceral progression as defined by RECIST v1.1 with PCWG3 modifications * Appearance of ≥ 2 new lesions in bone scan * Have been treated with ≥ 1 second-generation androgen-signaling inhibitor, including abiraterone, apalutamide, darolutamide, and/or enzalutamide * Have been treated with ≥ 1 prior taxane regimens (example (e.g.), docetaxel, cabazitaxel) in either mHSPC or mCRPC setting * Participants with known mutations in BRCA1/2 must have received a PARP inhibitor if an approved therapy is locally available, and participant is considered suitable for this therapy * Are deemed unsuitable for standard of care (Only applicable for Part 1) Inclusion Criteria Applicable for PSMA-Targeted Radioligand-Experienced mCRPC Cohort Only (Part 2 Cohort 2-1) * Have metastatic disease, defined by ≥ 1 metastatic lesion that is present on baseline CT, MRI, or bone scan imaging * Have documented progressive disease (PD) based on ≥ 1 of the following criteria (per PCWG3): * PSA level ≥ 1 ng/mL that has increased on ≥ 2 successive occasions ≥ 1 week apart * Nodal or visceral progression, as defined by RECIST v1.1 with PCWG3 modifications * Appearance of ≥ 2 new lesions in bone scan * Have had prior bilateral orchiectomy and/or are receiving ADT and have a castrate level of serum testosterone (\< 50 ng/dL or \< 1.7 nmol/L) * Have received PSMA-targeted radioligand-based therapy, such as 177Lu-PSMA-617 in any setting * Have been treated with ≥ 1 second-generation androgen-signaling inhibitor, including abiraterone, apalutamide, darolutamide, and/or enzalutamide * Have been treated with ≥ 1 prior taxane regimens (e.g., docetaxel, cabazitaxel) in either mHSPC or mCRPC setting * Participants with known mutations in BRCA1/2 must have received a PARP inhibitor if an approved therapy is locally available, and participant is considered suitable for this therapy Inclusion Criteria Applicable for 1L/2L Taxane-Naïve mCRPC Cohorts Only (Cohort 2-3, Part 3, Cohort 4a-1, and Cohort 4b-1) * Prior treatment with ARSI (including but not limited to abiraterone, enzalutamide, darolutamide, or apalutamide). Participants without prior ARSI treatment in the mCRPC setting are also allowed * No prior exposure to taxane-based chemotherapy or other cytotoxic agents, including antibody-drug conjugates (ADCs) administered specifically for the treatment of mCRPC * Participants with known mutations in BRCA1/2 must have received a PARP inhibitor if an approved therapy is locally available, and participant is considered suitable for this therapy * Have metastatic disease, defined by ≥ 1 metastatic lesion that is present on baseline CT, MRI, or bone scan imaging * Have documented PD based on ≥ 1 of the following criteria (per PCWG3): * PSA level ≥ 1 ng/mL that has increased on ≥ 2 successive occasions ≥ 1 week apart * Nodal or visceral progression, as defined by RECIST v1.1 with PCWG3 modifications * Appearance of ≥ 2 new lesions in bone scan * Have had prior bilateral orchiectomy and/or are receiving ADT and have a castrate level of serum testosterone (\< 50 ng/dL or \< 1.7 nmol/L) Inclusion Criteria Applicable for mHSPC Cohorts Only (Part 3, Cohort 4a-2, and Cohort 4b-2) * Histologically documented prostate adenocarcinoma which is de novo or recurrent and considered castration-sensitive * Metastatic disease prior to first dose, with clear evidence of ≥ 1 bone lesion (defined as one lesion with positive uptake on bone scan) and/or ≥ 1 soft tissue lesion (measurable and/or non-measurable) that can be accurately assessed at baseline and is suitable for repeated assessment with CT and/or MRI. Participants with metastatic disease identified by PSMA-PET only, will not be eligible. Participants with disease limited to regional pelvic lymph nodes only are not eligible * Participants should have high-risk OR high-volume disease per the following criteria: * High-risk is defined as presence of at least 2 of the following: Gleason score of greater than or equal to (≥) 8; presence of 3 or more lesions on bone scan; presence of measurable visceral (excluding lymph node disease) metastasis on CT or MRI * High-volume disease is defined as: Presence of visceral metastases OR 4 bone lesions, with ≥1 beyond the vertebral bodies and pelvis * Participant is receiving ADT with a GnRH analogue or has undergone bilateral orchiectomy starting ≥ 14 days and \< 4 months prior to first dose with no radiographic evidence of disease progression or rising PSA levels prior to first day of dosing Inclusion Criteria Applicable for High-Risk BRPC Cohorts Only (Dose Optimization, Part 3a, and Part 4a-3) * Have BRPC defined by meeting all of the following criteria: * Prostate cancer initially treated by radical prostatectomy or radiotherapy (including brachytherapy) or both, with curative intent * PSA doubling time ≤ 12 months at the time of study entry * Screening PSA ≥ 1 ng/mL in patients who have had radical prostatectomy (with or without radiotherapy) as primary treatment for prostate cancer and at least 2 ng/mL above the nadir in patients who had radiotherapy only as primary treatment for prostate cancer * Screening serum testosterone \> 150 ng/dL Inclusion Criteria Specific to Part 1a (Japanese Participants with mCRPC) * Participant is of Japanese descent and will be participating at a site in Japan * Have metastatic disease, defined by ≥ 1 metastatic lesion that is present on baseline CT, MRI, or bone scan imaging * Have had prior bilateral orchiectomy and/or are receiving an ADT and have a castrate level of serum testosterone (\< 50 ng/dL or \< 1.7 nmol/L) * Have documented progressive mCRPC based on ≥ 1 of the following criteria (per PCWG3: * PSA level ≥ 1 ng/mL that has increased on ≥ 2 successive occasions ≥ 1 week apart * Nodal or visceral progression as defined by RECIST v1.1 with PCWG3 modifications * Appearance of ≥ 2 new lesions in bone scan * Have been treated with ≥ 1 second-generation androgen-signaling inhibitor, including but not limited to abiraterone, apalutamide, darolutamide, and/or enzalutamide * Have been treated with ≥ 1 prior taxane regimens (e.g., docetaxel, cabazitaxel) in either the mHSPC or mCRPC setting * Prior treatment with PSMA-targeted radioligand therapy, immunotherapy, sipuleucel-T, PARP inhibitors, and radium-223 is allowed * Participants with known mutations in BRCA1/2 must have received a PARP inhibitor if an approved therapy is locally available, and participant is considered suitable for this therapy Inclusion Criteria Specific to Part 2a (Chinese Participants with mCRPC) * Participant is of Chinese descent and will be participating at a site in China * Have metastatic disease, defined by ≥ 1 metastatic lesion that is present on baseline CT, MRI, or bone scan imaging * Have had prior bilateral orchiectomy and/or are receiving ADT and have a castrate level of serum testosterone (\< 50 ng/dL or \< 1.7 nmol/L) * Have documented progressive mCRPC based on ≥ 1 of the following criteria (per PCWG3): * PSA level ≥ 1 ng/mL that has increased on ≥ 2 successive occasions ≥ 1 week apart * Nodal or visceral progression as defined by RECIST v1.1 with PCWG3 modifications * Appearance of ≥ 2 new lesions in bone scan * Have been treated with ≥ 1 second-generation androgen-signaling inhibitor, including but not limited to abiraterone, apalutamide, darolutamide, and/or enzalutamide * Have been treated with ≥ 1 prior taxane regimens (e.g., docetaxel, cabazitaxel) in either the mHSPC or mCRPC setting * Prior treatment with PSMA-targeted radioligand therapy, immunotherapy, sipuleucel-T, PARP inhibitors, and radium-223 is allowed * Participants with known mutations in BRCA1/2 must have received a PARP inhibitor if an approved therapy is locally available, and participant is considered suitable for this therapy Exclusion Criteria: * Have dominant histopathological features representative of sarcomatoid, spindle cell, or neuroendocrine small cell components * Have acute or chronic infections * Have a concomitant medical or inflammatory condition that may increase the risk of toxicity to VIR-5500 (AMX-500) * Have known active CNS metastases and/or carcinomatous meningitis * Have lesions in proximity of vital organs Applicable to mHSPC cohorts only: * Any prior treatment with a GnRH analogue or ARSI * Any prior anticancer pharmacotherapy or surgery for metastatic prostate cancer Exclusion Criteria Applicable to Part 3a and Part 4a Only: VIR-5500 (AMX-500) in Combination With an ARSI (Enzalutamide or Darolutamide) * Are progressing rapidly while taking a second-generation androgen-signaling inhibitor * Have a history or presence of clinically relevant CNS pathology within 12 months of enrollment * Have had significant cardiovascular disease within the last 3 months Exclusion Criteria Applicable to Part 3b and Part 4b Only: VIR-5500 (AMX-500) in Combination With Docetaxel ± ARSI * Abnormal liver function test (LFT) values during screening * Currently receiving either strong or moderate inhibitors of cytochrome P450 (CYP)3A4 that cannot be discontinued for the duration of the study