Gana Gana - 1 month ago 7
Java Question

Add a file inside a folder to Github using JGit or EGit - directory1\myfile.txt

*Added final working program to the bottom of this question *

I was able to add a file to GitHub using org.eclipse.egit.github.core java library (and code sample is referred from here: https://gist.github.com/Detelca/2337731 )

But I am not able to give a path like "folder1\myfile.txt" OR "folder1\folder2\myfile.txt"

I am trying to find a simple example to add a file under a folder and was not really able to find it.

Any help on this with an example?

Below is the code i am using: (configuration details like username, reponame are in the method addFileToGH() )

import java.io.IOException;
import java.util.ArrayList;
import java.util.Calendar;
import java.util.Collection;
import java.util.Date;
import java.util.List;

import org.eclipse.egit.github.core.Blob;
import org.eclipse.egit.github.core.Commit;
import org.eclipse.egit.github.core.CommitUser;
import org.eclipse.egit.github.core.Reference;
import org.eclipse.egit.github.core.Repository;
import org.eclipse.egit.github.core.RepositoryCommit;
import org.eclipse.egit.github.core.Tree;
import org.eclipse.egit.github.core.TreeEntry;
import org.eclipse.egit.github.core.TypedResource;
import org.eclipse.egit.github.core.User;
import org.eclipse.egit.github.core.client.GitHubClient;
import org.eclipse.egit.github.core.service.CommitService;
import org.eclipse.egit.github.core.service.DataService;
import org.eclipse.egit.github.core.service.RepositoryService;
import org.eclipse.egit.github.core.service.UserService;

public class GHFileWriter {

public static void main(String[] args) {
try {
new GHFileWriter().addFileToGH("myfile.txt", "some file content");
//new GHFileWriter().addFolderToGH("folder1");
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}


public boolean addFileToGH(String fileName, String fileContent) throws IOException{
String userName = "myusername";
String password = "mypassword";
String repoName = "myrepository";
String ownerName = "myusername";
String email = "myemail@gmail.com";

//Basic authentication
GitHubClient client = new GitHubClient();
client.setCredentials(userName, password);

// create needed services
RepositoryService repositoryService = new RepositoryService();
CommitService commitService = new CommitService(client);
DataService dataService = new DataService(client);

// get some sha's from current state in git
Repository repository = repositoryService.getRepository(ownerName, repoName );
String baseCommitSha = repositoryService.getBranches(repository).get(0).getCommit().getSha();
RepositoryCommit baseCommit = commitService.getCommit(repository, baseCommitSha);
String treeSha = baseCommit.getSha();

// create new blob with data
Blob blob = new Blob();
blob.setContent( fileContent ).setEncoding(Blob.ENCODING_UTF8); //"[\"" + System.currentTimeMillis() + "\"]").setEncoding(Blob.ENCODING_UTF8);
String blob_sha = dataService.createBlob(repository, blob);
Tree baseTree = dataService.getTree(repository, treeSha);
Collection<TreeEntry> entries = new ArrayList<TreeEntry>();

// create new tree entry
TreeEntry treeEntry = new TreeEntry();
treeEntry.setPath( fileName );
treeEntry.setMode(TreeEntry.MODE_BLOB);
treeEntry.setType(TreeEntry.TYPE_BLOB);
treeEntry.setSha(blob_sha);
treeEntry.setSize(blob.getContent().length());

/* //Code for adding folder, not working
TreeEntry folderEntry = new TreeEntry();
folderEntry.setPath( "folder1" );
folderEntry.setMode(TreeEntry.MODE_DIRECTORY);
folderEntry.setType(TreeEntry.TYPE_TREE);
folderEntry.setSha(blob_sha); //one of the possible issues
//folderEntry.setSize(blob.getContent().length());

entries.add(folderEntry);

*/
entries.add(treeEntry);

Tree newTree = dataService.createTree(repository, entries, baseTree.getSha()); //Issue is popped-up here, error is thrown here

// create commit
Commit commit = new Commit();
commit.setMessage("Test commit at " + new Date(System.currentTimeMillis()).toLocaleString());
commit.setTree(newTree);



//Due to an error with github api we have to to all this
//TODO: Make this better (another function)
UserService userService = new UserService(client);
User user = userService.getUser();

CommitUser author = new CommitUser();
Calendar now = Calendar.getInstance();

author.setName( userName );
author.setEmail(email);
//author.setEmail(userService.getEmails().get(0));
author.setDate(now.getTime());
commit.setAuthor(author);
commit.setCommitter(author);


List<Commit> listOfCommits = new ArrayList<Commit>();
listOfCommits.add(new Commit().setSha(baseCommitSha));
// listOfCommits.containsAll(base_commit.getParents());
commit.setParents(listOfCommits);
// commit.setSha(base_commit.getSha());
Commit newCommit = dataService.createCommit(repository, commit);

// create resource
TypedResource commitResource = new TypedResource();
commitResource.setSha(newCommit.getSha());
commitResource.setType(TypedResource.TYPE_COMMIT);
commitResource.setUrl(newCommit.getUrl());

//System.out.println("Committed file URL: "+ newCommit.getUrl());
// get master reference and update it
Reference reference = dataService.getReference(repository, "heads/master");
reference.setObject(commitResource);
dataService.editReference(repository, reference, true);

if( newCommit.getUrl() != null || !newCommit.getUrl().equalsIgnoreCase("") ){
return true;
}
//push.setCommits(commits);

return false;
}
}


Below is the error when i execute the above code.


org.eclipse.egit.github.core.client.RequestException: tree.sha
a3b93733a6dfd221c24e94d2ce52c25307910d73 is not a valid tree (422) at
org.eclipse.egit.github.core.client.GitHubClient.createException(GitHubClient.java:552)
at
org.eclipse.egit.github.core.client.GitHubClient.sendJson(GitHubClient.java:643)
at
org.eclipse.egit.github.core.client.GitHubClient.post(GitHubClient.java:757)
at
org.eclipse.egit.github.core.service.DataService.createTree(DataService.java:203)
at
com.apps.ui5.accelerator.writers.GHFileWriter.addFileToGH(GHFileWriter.java:87)
at
com.apps.ui5.accelerator.writers.GHFileWriter.main(GHFileWriter.java:30)


Edit: The error causing code is commented. The commented code starts with "//Code for adding folder, not working"

Possible issue is with this line of code: folderEntry.setSha(blob_sha);

The error is thrown at this line: Tree newTree = dataService.createTree(repository, entries, baseTree.getSha());




Edit(VonC response): Hi VonC, thanks a lot for your response.
JGit looks promising but i see below issues.


  1. I am running this as a service in a cloud instance where i dont know if i can rely on filerepo which is refererred in the cookbook example. If this works in cloud, i can consider that as a solution.

  2. The example is also adding a simple file but not a folder/directory.

  3. And I need to push changes to GitHub using a java program, not git commands. How can i do that?



Can you refer any example which is creating a folder? I tried below code for adding a folder but did not work!

import java.io.File;
import java.io.IOException;
import java.io.PrintWriter;

import org.dstadler.jgit.helper.CookbookHelper;
import org.eclipse.jgit.api.Git;
import org.eclipse.jgit.api.errors.GitAPIException;
import org.eclipse.jgit.api.errors.NoFilepatternException;
import org.eclipse.jgit.lib.Repository;


public class GHFolderWriter {

public static void main(String[] args) throws IOException, NoFilepatternException, GitAPIException {
// prepare a new test-repository
try (Repository repository = CookbookHelper.createNewRepository()) {
try (Git git = new Git(repository)) {
//create folder
File folder = new File(repository.getDirectory().getParent(), "folder1");
if(!folder.createNewFile()) {
throw new IOException("Could not create folder " + folder);
}

// create the file
File myfile = new File(folder, "testfile1.txt");
if(!myfile.createNewFile()) {
throw new IOException("Could not create file " + myfile);
}

// Stage all files in the repo including new files
git.add().addFilepattern(".").call();

// and then commit the changes.
git.commit()
.setMessage("Commit all changes including additions")
.call();

try(PrintWriter writer = new PrintWriter(myfile)) {
writer.append("Hello, world!");
}

// Stage all changed files, omitting new files, and commit with one command
git.commit()
.setAll(true)
.setMessage("Commit changes to all files")
.call();


System.out.println("Committed all changes to repository at " + repository.getDirectory());
}
}

}

}


It is throwing this error:


Exception in thread "main" java.io.IOException: Could not open file.
Reason : The system cannot find the path specified (path
C:\Users\ramgood\AppData\Local\Temp\TestGitRepository441020326444846897\folder1\testfile1.txt,
working dir C:\Users\ramgood\workspace-sapui5-1.32\accelerate) at
java.io.WinNTFileSystem.createFileExclusively(Native Method) at
java.io.File.createNewFile(File.java:1016) at
com.apps.ui5.accelerator.writers.GHFolderWriter.main(GHFolderWriter.java:37)


Edit(3): I moved to jgit as you suggested and below is my new program. I am able to 1. clone remote repo with my local repo 2. add a folder and file to local repo and commit locally but 3. Failing to push to github.

import java.io.File;
import java.io.IOException;
import java.io.PrintWriter;

import org.eclipse.jgit.api.Git;
import org.eclipse.jgit.api.errors.GitAPIException;
import org.eclipse.jgit.lib.Repository;
import org.eclipse.jgit.storage.file.FileRepositoryBuilder;
import org.eclipse.jgit.transport.UsernamePasswordCredentialsProvider;



/**
* Note: This snippet is not done and likely does not show anything useful yet
*
* @author dominik.stadler at gmx.at
*/
public class PushToRemoteRepository {

private static final String REMOTE_URL = "https://github.com/debsap/testrepo.git";

public static void main(String[] args) throws IOException, GitAPIException {
// prepare a new folder for the cloned repository
File localPath = File.createTempFile("REStGitRepository", "");
if(!localPath.delete()) {
throw new IOException("Could not delete temporary file " + localPath);
}

// then clone
System.out.println("Cloning from " + REMOTE_URL + " to " + localPath);
try (Git git = Git.cloneRepository()
.setURI(REMOTE_URL)
.setCredentialsProvider(new UsernamePasswordCredentialsProvider("debsap", "testpasword1"))
.setDirectory(localPath)
.call()) {

Repository repository = git.getRepository();
// create the folder
File theDir = new File(repository.getDirectory().getParent(), "dir1");
theDir.mkdir();

// create the file
File myfile = new File(theDir, "testfile2.txt");
if(!myfile.createNewFile()) {
throw new IOException("Could not create file " + myfile);
}

// Stage all files in the repo including new files
git.add().addFilepattern(".").call();

// and then commit the changes.
git.commit().setMessage("Commit all changes including additions").call();

try(PrintWriter writer = new PrintWriter(myfile)) {
writer.append("Hello, world!");
}
// Stage all changed files, omitting new files, and commit with one command
git.commit()
.setAll(true)
.setMessage("Commit changes to all files")
.call();
// now open the created repository
FileRepositoryBuilder builder = new FileRepositoryBuilder();
try (Repository repository1 = builder.setGitDir(localPath)
.readEnvironment() // scan environment GIT_* variables
.findGitDir() // scan up the file system tree
.build()) {

try (Git git1 = new Git(repository1)) {
git1.push().call();
}
System.out.println("Pushed from repository: " + repository1.getDirectory() + " to remote repository at " + REMOTE_URL);
}
}
}
}


I am getting below error while trying to push to remote repo or github.


Cloning from https://github.com/debsap/testrepo.git to
C:\Users\ramgood\AppData\Local\Temp\REStGitRepository8321744366017013430
Exception in thread "main"
org.eclipse.jgit.api.errors.TransportException: origin: not found. at
org.eclipse.jgit.api.PushCommand.call(PushCommand.java:183) at
org.dstadler.jgit.unfinished.PushToRemoteRepository.main(PushToRemoteRepository.java:88)
Caused by: org.eclipse.jgit.errors.NoRemoteRepositoryException:
origin: not found. at
org.eclipse.jgit.transport.TransportLocal$1.open(TransportLocal.java:131)
at
org.eclipse.jgit.transport.TransportBundleFile$1.open(TransportBundleFile.java:106)
at org.eclipse.jgit.transport.Transport.open(Transport.java:565) at
org.eclipse.jgit.transport.Transport.openAll(Transport.java:383) at
org.eclipse.jgit.api.PushCommand.call(PushCommand.java:147) ... 1
more


Below is the config file from .git folder. Note that i cannot edit this manually and should be updated through java program only.


[core] symlinks = false repositoryformatversion = 0 filemode =
false logallrefupdates = true [remote "origin"] url =
https://github.com/debsap/testrepo.git fetch =
+refs/heads/:refs/remotes/origin/ [branch "master"] remote = origin merge = refs/heads/master


Edit(Solution):
Below is the working program.

public void pushToRemote(String xmlViewContent) throws IOException, InvalidRemoteException, TransportException, GitAPIException{
// prepare a new folder for the cloned repository
File localPath = File.createTempFile("GitRepository", "");
if(!localPath.delete()) {
throw new IOException("Could not delete temporary file " + localPath);
}

// then clone
System.out.println("Cloning from " + REMOTE_URL + " to " + localPath);
try (Git git = Git.cloneRepository()
.setURI(REMOTE_URL)
.setCredentialsProvider(new UsernamePasswordCredentialsProvider(username, password))
.setDirectory(localPath)
.call()) {

Repository repository = git.getRepository();
// create the folder
File theDir = new File(repository.getDirectory().getParent(), "webapp");
theDir.mkdir();

// create the file
File myfile = new File(theDir, "InputView.view.xml");
myfile.createNewFile();

// Stage all files in the repo including new files
git.add().addFilepattern(".").call();

// and then commit the changes.
git.commit().setMessage("Commit all changes including additions").call();

try(PrintWriter writer = new PrintWriter(myfile)) {
writer.append( xmlViewContent );
}
// Stage all changed files, omitting new files, and commit with one command
git.commit()
.setAll(true)
.setMessage("Commit changes to all files")
.call();
git.add().addFilepattern("*").call();
RevCommit result = git.commit().setMessage("initial commit").call();
git.push()
.setCredentialsProvider(new UsernamePasswordCredentialsProvider(username, password))
.call();
System.out.println("Pushed with commit: "+ result);

}
}


Thanks to VonC for his responses.

Answer Source

For manipulating files in a Git repo, you might be better off using JGit directly, not Egit-GitHub (though it is based on JGit).

That way, you can benefit from the examples in centic9/jgit-cookbook, including a jgit/porcelain/CommitAll.java example using addFilepattern:

// Stage all files in the repo including new files
git.add().addFilepattern(".").call();

That way, you can add any folder (content) you need.

I am running this as a service in a cloud instance where I dont know if I can rely on filerepo which is refererred in the cookbook example. If this works in cloud, i can consider that as a solution.

You will rely on the same file repo you are using right now.

The example is also adding a simple file but not a folder/directory.

Simply replace the file by a folder: it will work and add the folder content.

And I need to push changes to GitHub using a java program, not git commands. How can I do that?

Simply by adding a remote to the GitHub repo, and pushing.
Both are available in JGit (so in Java).