{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Lecture 1b: Version control with Git\n", "\n", "Keeping track of all of the changes in your project over time is good practice. How many times have you edited something in something you were writing and then wanted to go back and see what you had in the first place? Wouldn't it be great to know what changes you made and when you made them?\n", "\n", "A **version control system** facilitates this process of keeping track of changes over time. Beyond that, it allows multiple people to collaborate and work on parts of the same project simultaneously. We will use version control extensively in class, both for collaboration among teammates, and for homework submission and grading.\n", "\n", "There are many version control systems. The four most prominent, in order of age, oldest to youngest, are [CVS](https://en.wikipedia.org/wiki/Concurrent_Versions_System), [Subversion](https://subversion.apache.org), [Git](https://git-scm.com), and [Mercurial](https://www.mercurial-scm.org) (the first version of Git was released about two weeks before Mercurial, so they are really the same age). Today, Git dominates.\n", "\n", "Git was developed by [Linus Torvalds](https://en.wikipedia.org/wiki/Linus_Torvalds), the person who developed the Linux operating system. He named Linux after himself, and he decided to also name Git after himself (\"git\" is British slang for a stupid person). Try typing\n", "\n", "```bash\n", "man git\n", "```\n", " \n", "on the command line and read what the `NAME` of the software is.\n", "\n", "Using Git as a version control system allows communication with remote repositories (\"repos\" for short) hosted by services such as [GitHub](http://github.com/), [GitLab](http://gitlab.com/) or [Bitbucket](http://bitbucket.org). These services provide university-affiliated people with a `.edu` email address with perks that include free private repositories. We will use GitHub for our bootcamp, and you should already have set up an account.\n", "\n", "Remote repositories are not only a great way for keeping your data safe and in the cloud. They are also an excellent tool for collaboration since Git allows multiple users to edit the shared files simultaneously and has a method to merge changes afterwards. Public repositories can also serve as a vehicle to distribute code (or other files).\n", "\n", "You can find more information about Git [here](https://git-scm.com/documentation). It is well documented. Here is an [excellent one-page (front-and-back) cheatsheet](https://education.github.com/git-cheat-sheet-education.pdf).\n", "\n", "Let's get started. You all should have a version of Git installed on your computers. Open the terminal and navigate into your `~/git` directory." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Configuring Git\n", "\n", "While you already used Git in the set-up lessons, you still have some configuring to do. For this, and everything else in this notebook, we'll use the command line. We will do the configuration with `--global` flags, which means these specifications work for all of your repositories. First, we'll specify the name and email address of the person working with Git on your machine (that's you!).\n", "\n", "```bash\n", "git config --global user.name \"YOUR NAME\"\n", "git config --global user.email \"YOUR EMAIL ADDRESS\"\n", "```\n", "\n", "\n", "\n", " \n", "Git is very well documented and help is easily available. If you need to know more about `config`, for example, just enter:\n", "\n", "```bash\n", "git help config\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Cloning Repositories\n", "\n", "Your team's repository is already set up and hosted on GitHub. You will need to clone it from its URL; you will have access to that once the teams are finalized. To practice cloning, we will clone one of the zillions of public repositories that are hosted on GitHub. We will clone a simple package, called `insulter` that will hurl Shakespearean insults at you. Enter the following on the command line.\n", "\n", "```bash\n", "git clone https://gist.github.com/3165396.git insulter\n", "```\n", "\n", "Note that the insulter package is now on your local machine. You have a copy of it on your own hard drive. You do not need to be connected to the internet to use it.\n", "\n", "Now, `cd` to `insulter` and you can start using it, thou wayward tickle-brained flap-dragon!\n", "\n", "```bash\n", "python insulter.py\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Pulling in changes\n", "\n", "Actively developed repositories are constantly being updated. After you clone the repository, its authors (e.g., your teammates) may add or edit things in the repository. For you to get those changes, you need to **fetch** them and then **merge** them into what you have locally.\n", "\n", "To fetch the updated repository, you guessed it, you do:\n", "\n", "```bash\n", "git fetch\n", "```\n", "\n", "The result is stored in a hidden directory, `.git/FETCH_HEAD`. (Directories that begin with a `.` are hidden; you don't see them when you type `ls` without any flags.)\n", "\n", "Now that there are changes, you would like to update your local repository. Provided you do not have any local edits, this is seamless. You just do\n", "\n", "```bash\n", "git merge FETCH_HEAD\n", "```\n", " \n", "Now your repository will be up to date.\n", "\n", "A shortcut for the commands\n", "\n", "```bash\n", "git fetch\n", "git merge FETCH_HEAD\n", "```\n", " \n", "run in succession is\n", "\n", "```bash\n", "git pull\n", "```\n", " \n", "In practice, you will use this a lot, but sometimes fetching and merging can be useful. Note that `git pull` is actually shorthand for \n", "\n", "```bash\n", "git pull origin master\n", "```\n", " \n", "which is the more verbose way of saying that you want to pull the master **branch** from the remote repository named `origin`. We will not discuss branching in this course, but it is an important concept to learn about.\n", "\n", "Generally it is good practice to pull before you start working each day to make sure you pull in any updates your collaborators may have made." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Relationships among your local machine and your repository\n", "\n", "Let us think about how we have things set up, referring to the excellent image below from [Jessica Lord](http://jlord.us/). For now, focus only on the blue portions on the right (ignore forking and the upstream repository). **Remotely**, in our case on GitHub, lives the repository that I created for your team.\n", "\n", "\n", "
\n", "\n", "
\n", " \n", "![Jessica Lord Git diagram](jessica_lord_git_diagram.png)\n", "\n", "
\n", " \n", "
\n", "\n", "When you **cloned** your repository, you pulled it down from GitHub and put it on your **local** machine (probably your laptop). Whenever you make changes the repository, you do so on your local machine. You then **add** whatever files you changed to alert Git that you care about the changes and want it to keep them under version control. You then **commit** your changes, which tells Git to create a snapshot of exactly what the repository looked like when you did the commit. Associated with a commit is a **commit message**, which briefly describes what you changed on that commit. When you **push** your commits, they go to your team's repository. \n", "\n", "So, here is how the command line would look if you wanted to work on a file called `my_file.py` in your repository. First, pull in any possible updates from your teammates. Actually, let me say this more forcefully.\n", "\n", "
\n", "\n", "
Always `git pull` when you start working.
\n", "\n", "
\n", "\n", "The first thing you do on the command line is\n", "\n", "```bash\n", "git pull\n", "```\n", "\n", "Now that you have an updated repository to work with, you go to work and you edit `my_file.py`, let say by cleaning up some of the data wrangling procedures. You then need to tell Git add the file.\n", "\n", "```bash\n", "git add my_file.py\n", "```\n", "\n", "You might want to check to make sure you didn't have other edits you want to add to the commit. You can do this by checking the status.\n", "\n", "```bash\n", "git status\n", "```\n", "\n", "The result might look something like this.\n", "\n", "```\n", "On branch master\n", "Your branch is up to date with 'origin/master'.\n", "\n", "Changes to be committed:\n", " (use \"git reset HEAD ...\" to unstage)\n", "\n", " modified: my_file.py\n", "\n", "\n", "Changes not staged for commit:\n", " (use \"git add ...\" to update what will be committed)\n", " (use \"git checkout -- ...\" to discard changes in working directory)\n", "\n", " modified: another_file.py\n", "\n", "Untracked files:\n", " (use \"git add ...\" to include in what will be committed)\n", "\n", " yet_another_file.py\n", "\n", "no changes added to commit (use \"git add\" and/or \"git commit -a\")\n", "```\n", "\n", "In that status report from Git, it says the you modified a file called `another_file.py` that is already under version control (it has been added and included in a commit before), but you have not added it for this commit. If you want to add it, you would do\n", "\n", "```bash\n", "git add another_file.py\n", "```\n", "\n", "There is another file, `yet_another_file.py` that exists in the directory that you have never added, so Git is not tracking it. You can also add that for this commit.\n", "\n", "```bash\n", "git add yet_another_file.py\n", "```\n", "\n", "Now everything is ready to commit. To commit with a commit message, do the following.\n", "\n", "```bash\n", "git commit -m \"Updated wrangling methods in my_file.py.\"\n", "```\n", "\n", "Git now has a snapshot of exactly what your repository looked like at the time of the commit. Now, the commit is still only on your local machine. In order for your collaborators (or the whole world, if it is a public repo) to have access to it (and in order for it to appear on GitHub), you need to **`push`** it. To do that, we do this:\n", "\n", "```bash\n", "git push origin master\n", "```\n", "\n", "Here, `origin` is a nickname for your remote repository. `master` is the name of the branch we are pushing to in the GitHub repository. I.e., it is the master copy. (We will not really talk about branches or branching.)\n", "\n", "That is the basic set of operations to get your work into the repository and out to the remote for your teammates." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### A note on commit messages\n", "\n", "As just described, the `-m` after `git commit` specifies the **commit message**. This is a brief bit of text that describes what has changed in the repository. It is really important to write informative commit messages (and [here are some beautifully described rules for good commit messages](https://chris.beams.io/posts/git-commit/)). By contrast, most [commit messages that people write are useless](http://ramiro.org/blog/most-frequent-github-commit-messages/). \n", "\n", "Note that good commit messages also require that you commit often for incremental changes. Good commit messages is unfortunately one of my greatest weaknesses. But it should not be yours." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### What to do in case of a fire\n", "\n", "It is important to commit and push often. It helps collaboration and protects your work. So, what do you do in case of a fire in your building?\n", "\n", "```bash\n", "git add *\n", "git commit -m \"Building on fire, no time for meaningful commit message.\"\n", "git push origin master\n", "git out of the building\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## .gitignore\n", "\n", "Notice in the sample output of `git status` above, Git let us know that there was an untracked file in the directory. Sometimes you do want to have files in the directories of your repository, but not keep those files under version control. Examples of these might be binary files, large data sets, images, etc.\n", "\n", "Fortunately, you can tell Git to ignore certain files. This is done using a [.gitignore file](https://git-scm.com/docs/gitignore). Each line of of the `.gitignore` file says which files to ignore. For example, to ignore all files that end with `.tif`, you would include the line\n", "\n", " *.tif\n", " \n", "in your `.gitignore` file. The `*` is a **wildcard character** which says to ignore all files that have a file name ending with `.tif`, regardless of what the prefix is. Now, whenever you you use `git status`, any file ending with `.tif` that happens to be on your machine within the directories containing your repository will be ignored by Git.\n", "\n", "Just because `*.tif` appears in a `.gitignore` file does not mean that *all* `.tif` files will be ignored. If you explicitly add a file to the repository, Git will keep track of it. E.g., if you did\n", "\n", " git add myfile.tif\n", "\n", "then `myfile.tif` will be under version control, even if other `.tif` files laying around are not. (*Note, though, that you typically do not want to have binary files under version control. You usually only keep code under control. Typically only data sets used to test code are included in the repository. Version control is not really for data, but for code.*)\n", "\n", "Finally, since it begins with a `.`. When you put a `.gitignore` file in a directory, the `.gitignore` file will not show up when you run `ls` at the command line without the `-a` flag.\n", "\n", "You have a `.gitignore` file already present in the repositories I set up for you. It is there to tell Git to ignore the files in the folder `data/`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Additional information about using Git\n", "\n", "This lecture gives a minimal introduction about Git/GitHub that you need for this class. You can get a bit more detailed view of this material from my bootcamp materials [here](http://justinbois.github.io/bootcamp/2019/lessons/l12_version_control_with_git.html). In recitation, the TAs will give you more tips and traps about using Git. Beyond that, there is much to learn about, in particular branching. For that and much more, there are plenty of great resources on the internet. GitHub has a good [series of guides](https://guides.github.com) to get you started." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Computing environment" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "CPython 3.7.3\n", "IPython 7.1.1\n", "\n", "jupyterlab 0.35.5\n" ] } ], "source": [ "%load_ext watermark\n", "%watermark -v -p jupyterlab" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.4" } }, "nbformat": 4, "nbformat_minor": 4 }