Managing multiple projects with common code

Starting this thread to discuss ideas for a tool (set of tools) to help manage sharing code between Xojo projects

Because of how Xojo uses manifests for Text projects keeping the manifest in sync is a chore to do manually and there’s no built in ways to do this

So what I’m thinking is a command line tool (maybe a UI on top of it as well) that can “pull” selected code from one text project (call it the master) into other projects (clients)

An initial pull would modify the manifest of the project where the code is pulled into by adding references to those files from the master (the code is not duplicated)

There may be options like

  • pull references (adds paths that refer to the original files but does not copy files)
  • pull code (actually pulls the code across into the client & can be "resented) at any time to bring over updated code

Thoughts ?

If I was going to blue sky something like you are suggesting I would really like “Common Code” to exist in some sort of identifiable container in the IDE that could be named, and when you pull it into a new app the items that need to be app specific connections, variables, etc. have a todo list to make connections in the new app. By design you can designate something as common code by putting code into a “Common Code Container” identify the common todo items and that makes it transportable.

I’m not sure that would be doable unless Xojo was cooperating to allow inserting a new type of container, (which in reality could perhaps be a folder or maybe a module with a different icon). Of course I have no idea what I’m talking about… I can sort of picture it, but I’m not sure I described what I see in my minds eye and that actually may have no relationship to what you are trying to accomplish. </crawls back under his rock>

Yeah I cant rewrite the IDE so it HAS to work with existing project formats & such

I could make the tool pull code into “Common Code” folder in whatever client projects pull code

The rest pulling references vs pulling in actual copies makes sense though ?

Yes. References makes sense but as an option, I think. If you pull code and modify it, could you put it back into a common code repository as an iteration/revision for other apps to pull as a reference?

since I have no control over what ever VCS a person might be using … maybe pushing it back

right now I’m thinking there’s a “master” and several “clients” that draw FROM the master
Mostly as a one way things where
Unless the reference form is used then its literally "the same code in all projects using the reference form since its just relative paths

What I normally do is keep common code in a separate folder(s) & repository, not “linked” to anything else. This is where I make development changes to it. Then I copy files from there to a shared common folder, that’s not in a repository, and then my projects reference the code from there (Import as external in Xojo). This way any unintentional changes don’t get replicated into the common code repository, and neither does the Xojo “cruft” that occurs everytime I save any project that references that code.

There are git hooks that could be utilized to handle some or all of this automatically, but I haven’t taken the time to try it. I have set up git hooks to do email notifications on private repositories* with my clients. Git has hooks that run on the client and others that run on the server.

*These private repositories are not on GitHub or any other cloud service. They are just headless repositories on a web server. Most web servers come with headless (command-line only) git available to use if you know how.

we DO want each of the to be the “parent project” + client projects
that use that common code

so in the parent we can fix bugs etc, maybe even add items to the shared code, and make a tool that quickly & simply can “pull” that code into client projects

git sub projects cant do that because of the manifest

we might have 50 such projects that use this common repo so doing it manually simply isn’t viable

Ah, just realized you’re talking about working around Xojo’s limitation of not being able to use text code files as external (right?). Currently they have to be XML or Binary. So you’re wanting to copy text code files from the common folder(s) into each project’s folder and update the manifest, which is necessary for Xojo to “see” them (this isn’t necessary for existing code files that are just updated, is it?).

Because of this limitation, I “save as XML or Binary” my common code to the shared folder I mentioned. Then it’s just a matter of ⌘+Option or Shift+Ctrl dragging new ones (not updated ones) to projects to create the links. Projects already linked just need to be reopened.

I don’t see why not. I haven’t really looked at the manifest file, but there are many ways to “wrangle” text files in the shell (for Mac and Linux, and now for Windows with the Linux subsystem installed). In case you didn’t know, git hook files are just shell scripts (bash, zsh, python, whatever), but with access to information about the repository (what was committed, pushed, pulled, etc.),. And you could create additional tools (console apps) as you’ve suggested. Then have git fire them off automatically.

Let me know if I’m understanding correctly, or if I’m still missing the crux of this issue. :confused: :man_shrugging:

ABSOLUTELY !
so it needs to be an external tool that can munge around existing project file sin a sensible fashion to make dealing with code easier

XML just sucks for VCS so I want to have an “original project” where bugs get fixed etc as plain text

then this tool is used to update other projects that USE the common code base

No I think you have it right

Not sure I want git to automatically update - yet
have to create the tool to do all this munging first I guess :slight_smile:

OK so if ANYONE is watching this topic, or cares, I am working on a prototype

Comments welcomed !

So far a “sync config” , which going in the same DIR as the project your going to sun other code INTO looks like

{ 
    "last-sync" : "" ,
    "sources" : [
        { "source" : "../project-dir/",
          "items" : [
            { "item" : "Folder=folder_full_of_parts" },    
            { "item" : "Class=timer_subclass" } ,           
            { "item" : "Module=some_module_name" }
          ]
        },
        { "source" : "../another-project/untitled.xojo_project",
          "items" : [
            { "item" : "Module=CSVModule" },    
            { "item" : "Module=generic_methods" }            
          ]
        }
    ]
}

last sync will get rewritten all the time with the last sync data (not 100% sold on this but … )

sources is an array of source projects so any number of other projects code can be copied in

each source has
source : a relative path to the dir containing a text project
OR
a relative path to a specific xojo_project file
an array of items where each item has
the KEY=VALUE name of the item from the manifest

the sync process will :

  1. verify the sources are reachable
  2. verify the items in each source are reachable
  3. verify that copying the items will not cause conflicts in terms of names, types, paths, ids etc

it will do all this BEFORE it ever touches the project its going to copy things in to
Because the process needs to be robust & IS going to alter project files etc it needs to be right before doing any of this (ESP if you’re NOT using source control for everything !)

Once it has verified that things are one it will

  1. copy ALL items from the source into the destination
  2. add every copied item to the manifest of the project that things re being copied into

If, as shown above, you copy a FOLDER from a project, you do not have to list all items contained in the folder. The process will copy them all automatically.

The same will happen for a module

You can list individual items IF you want and any needed paths will be added to the file system

Thoughts ?

FAQS :

  1. why copy and not use relative paths for everything ?
    relative paths only work well for some items
    and relative paths mean that any changes in the original immediately hap[pen in the other projects
    copies allow me to control WHEN the updates to other projects happen

  2. why is such tool needed ?
    You could copy & paste from one project to another
    But that doesnt scale very well, can be time consuming and, as noted in a couple bug reports, may not work for every kind of project item (web projects you cant copy the App instance or remove it so if you need to replace it you have to do something else)
    You could put everything in the same directory as others have done but that has its own set of risks when you intermingle many projects in the same directory. Some items, like Build Automation, don’t share well across many projects.

1 Like

well finally have working prototype and it works as I hoped !

perfect ? no

but a damn sight better than doing it by hand
resolves conflicting id’s between source projects and the target one
it doesnt fix duplicate item names
maybe it shouldn’t try to ?

The only thing I still need to sort out is what to do with deletions of items

say you incorporate all the items in a FOLDER from one project with a set up like the following which will copy EVERY item in “folder_full_of_parts” into the destination

{ 
    "last-sync" : "" ,
    "sources" : [
        { "source" : "../project-dir/",
          "items" : [
            { "item" : "Folder=folder_full_of_parts" },    
          ]
        }
    ]
}

If an item in the original is deleted I need to make sure that the destination project also removes that item from its manifest

So far this has NOT occurred with how I’m using it but could so I’ll need to sort that out soon