Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

more fixing tips #399

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/concepts/project/models.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,13 +52,13 @@ Open the HelloWorld_SQL Project. See the **(1)Lineage** for the HelloWorld_SQL P
![model-view](./img/models/model-view.png)
Now we can explore the `customers` Model more closely. The Model is easy to understand with interchangable **(1)visual** and **(2)code** views. The visual view depicts each small step needed to move from the referenced tables/seeds/Models to the final `customers` Model. Each transformation step or Common Table Expression (CTE) is called a **(3)Gem** in Prophecy.

The **(4)`Aggregate`** step is represented visually as an `Aggregate` Gem and in code as the highlighted CTE code fragment. By popular demand, the visual and code formats are editable interchangeably. Visual developers and SQL coders can work together in the same project, and both types of edits are incorporated to the project when [committed and merged](/docs/metadata/git.md#how-to-commit-changes).
The **(4)`Aggregate`** step is represented visually as an `Aggregate` Gem and in code as the highlighted CTE code fragment. By popular demand, the visual and code formats are editable interchangeably. Visual developers and SQL coders can work together in the same project, and both types of edits are incorporated to the project when [committed and merged](/docs/metadata/Git/git.md#how-to-commit-changes).

### Models vs Pipelines

If you’re already familiar with Prophecy Pipelines, Models are very similar. The major difference is that each Pipeline can create an arbitrary number of outputs, whereas a Model only defines one output. Where Pipelines can exist only within Spark-based projects, Models can exist within SQL-based ones.

Like Pipelines, Models can be configured, committed and released to [Git](/docs/metadata/git.md), according to software engineering best practices.
Like Pipelines, Models can be configured, committed and released to [Git](/docs/metadata/Git/git.md), according to software engineering best practices.

### dbt Core™ Models

Expand Down
2 changes: 1 addition & 1 deletion docs/deployment/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ There is no specific deployment step needed for other entities.

## How to Release a Project

Once you've [committed](/docs/metadata/git.md#how-to-commit-changes) and [merged](/docs/metadata/git.md#how-to-merge-changes) your changes, you're ready to release them. To do this, Simply proceed with the [Release and Deploy](/docs/metadata/git.md#how-to-release-a-branch), which takes care of both the release and deployment of Pipelines, Gems, and Jobs to respective environments.
Once you've [committed](/docs/metadata/Git/git.md#how-to-commit-changes) and [merged](/docs/metadata/Git/git.md#how-to-merge-changes) your changes, you're ready to release them. To do this, Simply proceed with the [Release and Deploy](/docs/metadata/Git/git.md#how-to-release-a-branch), which takes care of both the release and deployment of Pipelines, Gems, and Jobs to respective environments.

![Release_screen](img/release_and_deploy_screen.png)

Expand Down
6 changes: 6 additions & 0 deletions docs/metadata/Git/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"label": "Git",
"position": 2,
"collapsible": true,
"collapsed": true
}
57 changes: 57 additions & 0 deletions docs/metadata/Git/git-troubleshooting.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
---
title: Git Troubleshooting
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: are you okay with using sentence casing for titles and section headers? We have a mix of either that or title case in our docset, and I want to start enforcing one for consistency.

Git troubleshooting

Debugging Git credentials

Debugging network issues

id: git-troubleshooting
description: How to troubleshoot permissions and network issues with Git
tags: []
---

Sometimes there is an error when Prophecy tries to reach Git. This could be caused by your user's [authorization](#debugging-git-credentials), a [repository setting](#debugging-repository-setttings), or a [network](#debugging-network-issues) issue. Fear not! This guide outlines what to do if you see error messages such as:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should use second person, so "This could be cause by your [authorization]..."

`not authorized`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: to make it easier to read in this file, add empty spaces, so:

This guide outlines what to do if you see error messages such as:

not authorized
git-upload-pack-not-permitted

or

git-recieve-pack-not-permitted

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or better yet, use an unordered list for these, so:

  • not authorized
  • git-upload-pack-not-permitted
  • git-recieve-pack-not-permitted

`git-upload-pack-not-permitted`
or
`git-recieve-pack-not-permitted`

![errorMsgs](./../img/git_troubleshooting_1.png)

## Debugging Git Credentials

Sometimes your personal access token (PAT) has expired or lacks a certain permission.

1. Check if your PAT has expired. If so, create a new one with sufficient read and write access to the repository of interest. Here are the instructions to locate existing tokens and create new tokens for [Github](https://docs.github.com/en/enterprise-cloud@latest/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens), [Bitbucket](https://support.atlassian.com/bitbucket-cloud/docs/repository-access-tokens/), or [GitLab](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html#create-a-personal-access-token).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be mention where the reader can check to see if their PAT has expired?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it the Prophecy Access Tokens tab?

2. Open a terminal from a computer that can access your Git repository. If you need help with this step, please reach out to your Git administrator.
3. Have the new PAT ready; you'll use it in place of a password.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the reader doesn't have a new PAT, should we tell them where/how they can create one? Or link them to https://docs.prophecy.io/metadata/prophecyAPI/#generate-a-pat

4. Check for read access by testing your ability to [clone](https://git-scm.com/docs/git-clone) a repo
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For 4. and 5. I would add :'s at the end of the statements. And maybe use code blocks for better formatting and easier copy/paste-ability

`git clone`
Here are the clone instructions for [Github](https://docs.github.com/en/enterprise-cloud@latest/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#using-a-personal-access-token-on-the-command-line), [Bitbucket](https://support.atlassian.com/bitbucket-cloud/docs/clone-a-git-repository/), or [GitLab](https://docs.gitlab.com/ee/topics/git/clone.html).
5. Check for write access by testing your ability to [push](https://git-scm.com/docs/git-push) to a repo
`git push`
Here are the push instructions for [Github](https://docs.github.com/en/enterprise-cloud@latest/get-started/using-git/pushing-commits-to-a-remote-repository), [Bitbucket](https://support.atlassian.com/bitbucket-cloud/docs/push-code-to-bitbucket/), or [GitLab](https://docs.gitlab.com/ee/topics/git/commands.html#git-push).
6. If your PAT lacks read or write permissions for the repository of interest, ask the repository admin to adjust these permissions.
7. If access has changed recently, check with your Git administrator to see if they have made any changes.

## Debugging Repository Settings

Sometimes a repository settings can restrict a user's permissions outside the user's profile settings page.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"...restrict your permissions"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure where the reader would go to check the following restrictions. We could add a line saying something like, "On the repository's security settings, check for the following restrictions."


1. Check if there are restrictions on writing to certain branches.
2. Check if there are restrictions on commit hooks in the repository.
3. Check if there are restrictions on commit messages.
4. Check if the user's role allows read and write access on the repository.
5. If access has changed recently, check with your Git administrator to see if they have made any changes.

## Debugging Network Issues

If the PAT works outside the Prophecy UI as mentioned above but does not work inside the Prophecy UI then there may be network issues. If you're using app.Prophecy.io, then seek a network admin to troubleshoot why Prophecy cannot read or write from the repository. If you're using a private Prophecy deployment, follow the instructions below.

:::caution
The steps below are intended for Private SaaS deployments, i.e. Prophecy is deployed within your organization's network. The steps are intended for the person who owns the Prophecy deployment.
:::

1. Login to the metagraph pod.
2. If the remote repository is configured using SSL and the metagraph pod cannot reach the remote Git server, then set `SKIP_SSL_VERIFICATION` to true.
3. If Prophecy is running on an encrypted cluster, then `ENABLE_PROPHECY_ENCRYPTION_PII` must be set to true.
4. If the remote repository is configured via a proxy and the metagraph pod cannot reach the remote Git server, then add the proxy details to JVM options and restart.

## Reach out for assistance

If you're still facing issues, please don't hesitate to [reach out to us](/docs/getting-started/getting-help.md).
14 changes: 7 additions & 7 deletions docs/metadata/git.md → docs/metadata/Git/git.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: Git
id: Git
description: Source Control
sidebar_position: 2
sidebar_position: 1
tags:
- metadata
- Git
Expand All @@ -12,7 +12,7 @@ tags:

Configure your user's [Settings](https://app.prophecy.io/metadata/settings) with a new `Git` provider: enter `Email`, `Username` and `Personal Access Token`, or simply `Login with Github`.

![GitConnector](./img/git_connector.png)
![GitConnector](./../img/git_connector.png)

The video below walks through connecting to an external Git provider:

Expand All @@ -32,11 +32,11 @@ To grant authorization, follow the steps below:
1. Go to the [Authorized OAuth Apps settings page](https://github.com/settings/applications) on GitHub.
2. Look for our application named Prophecy in the list of authorized apps and click on the entry to view its details.

![GitOauthApps](img/git_oauth_apps.png)
![GitOauthApps](./../img/git_oauth_apps.png)

3. On the application details page, you should see an option to grant access to your organizations. Enable the necessary organization access permissions to allow our application to interact with your organization's repositories effectively. Please refer below image for permissions required.

![Permissions](img/oauth_permissions.png)
![Permissions](./../img/oauth_permissions.png)

After selecting the appropriate organization access permissions, click the Save or Authorize button to save your changes and grant access to our application.

Expand Down Expand Up @@ -112,7 +112,7 @@ Once the changes are merged, we can `release` a branch straight from the UI itse
<iframe src="https://user-images.Githubusercontent.com/103921419/174550916-7d8beb20-2013-401d-be30-67c02983958f.mp4" title="How to release a branch" allow="autoplay;fullscreen" allowtransparency="true" frameborder="0" scrolling="no" class="wistia_embed" name="wistia_embed" msallowfullscreen width="100%" height="100%"></iframe>
</div></div>

![Git after release](./img/git_release1.png)
![Git after release](./../img/git_release1.png)

:::note
Databricks scheduler changes would only come into effect once the Project is released.
Expand Down Expand Up @@ -142,7 +142,7 @@ When you fork a repository, you create a copy of the original repository (upstre

When using the fork per usage Git storage model while creating a new Project, apart from the `upstream repository` details, some additional information is needed for the `forked repository`.

![Fork repo setup](./img/fork_repo_eg1.png)
![Fork repo setup](./../img/fork_repo_eg1.png)

Let's create a fork of our original repository and pull `main` branch from original repository to our newly created forked repository.

Expand All @@ -154,7 +154,7 @@ Let's create a fork of our original repository and pull `main` branch from origi
Now to pull main branch from upstream repository or from origin of current repository is as simple as clicking on below
`Pull Upstream` or `Pull Origin` options

![Fork repo extra options](./img/fork_repo_eg2.png)
![Fork repo extra options](./../img/fork_repo_eg2.png)

:::note

Expand Down
Binary file added docs/metadata/img/git_troubleshooting_1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion docs/release_notes/2023/feb2023.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,7 @@ An administrator will have the option to create the following tables at the time

#### Seamless Git Integration with stored credentials

Added the ability to store [Git](/docs/metadata/git.md) credentials for a user across projects. Now the user can re-use Git credentials without re-authenticating.
Added the ability to store [Git](/docs/metadata/Git/git.md) credentials for a user across projects. Now the user can re-use Git credentials without re-authenticating.

#### Streaming Pipeline Support (_Beta_)

Expand Down
Loading