First of all, thanks for providing an excellent piece of software for free. PGO is exactly what I needed for at least part of what I'm trying to do.
I realize this question isn't completely relevant to PGO, but I saw that you provided help with similar problems so I'm hoping you'll be able to point me in the right direction here.
I'm trying to move a Yahoo group to bbPress. I have the messages and topics downloaded via PGO, which is the first step. bbPress includes an importer than can import a database from basically every other popular forum software, but not directly from Yahoo groups. So, I'm guessing the best method will be to convert the Yahoo data into a format that can be imported into one of the more popular forum softwares, and then import that database into bbPress.
The issue I'm having is how to convert the data I have into a format that can then be imported into another forum software. I basically have two related questions here:
1. Can I convert from .db3 into a format that can then be imported into something like phpBB or vBulletin?
2. If so, what format would that be, and what importers are available? I'm having trouble finding an importer for any of the major forums that will allow me to import from a raw format like .csv, or whatever I could get the PGO data converted to. It doesn't matter what forum software I use, since basically all of them can be imported through bbPress. It's just a matter of finding one that has an importer that will allow me to import the PGO data.
Hopefully that makes sense. Thanks in advance for any help you can provide.
I don't think there's going to be any other method than doing a bit of work to format the PGO data into a format that your importer of choice can consume.
It shouldn't be that hard. The basic data that any forum will require is
Author
Date
Time
Message subject
Message Text
Looking at your specific problem... bbPress doesn't seem to want a flat file input. It wants another forum format to leach from.
That means that if you want bbPress you will have to import the PGO data first to a forum bbPress can read.
I would suggest phpBB. I have some scripts to convert the old PGO .mdb archive to phpBB 3 (somewhere).
You'd just need to change the script that reads the .mdb database and change it from .mdb to .db3
Cheers,
Wilson.
Thanks for the advice. Would you be able to provide that script? That would help a lot. I'm assuming it would preserve the data you mentioned, so it would just be a matter of converting from .db3 -> phpBB database format (whatever that is) -> bbPress? As you said, if I could get the data into a database that functioned with phpBB3 then importing that using bbPress should be no problem.
I'll have to find those scripts. The last time I saw them was > 3 years ago.
They're fairly simple and there is a good set of instructions that go with them.
Anyway, give me a couple of days to find them...
Cheers,
Wilson.
No problem, I really appreciate the help.
OK, I have the scripts.
I will contact you via email regarding their use.
Cheers,
Wilson.
Update....
I now have a program to convert the PGO groups database to a format that can be consumed by the phpbb Message & User import mods.
Additionally, there is a plugin for bbPress to import phpBB data which I have tested and can verify works.
Cheers,
Wilson.
Would you be willing to license the PGO->phbb program? I know you offer a custom import service but for smaller yahoo groups the $100 price may be too much.
Sure I'll sell you cheap access to the phpBB converter but be aware, it'll still be like pulling teeth:
You will require:
1) A copy of XAMMP to create and run local copies of whatever forum software you decide to run + MySQL + Apache.
2) A working knowledge of PHP (assuming phpBB or Wordpress)
3) Good / very good knowledge of MySQL query language
4) A working knowledge of Python or C++.
5) Hosting for your forum.
I am writing up info for fellow YG moderators. It will help to tell them what your fee covers because I agree, it is worth every penny.
Ex:
"You can move your Group to WordPress if
1. You have a self-installed version of Wordpress on your own domain name
2. Know how to install plugins (ex: BBPress)
3. Have $100 to pay Wilson to convert your YG into a format that can be imported (for the first xx thousand messages, the cost for each xx thousand next messages is.....?????
For that fee all you have to do is ...??? give Wilson admin access to the group? or does he need to only be a member? give him admin access to your WordPress install? Does the moderator have to do anything else to get it up and running? Configure???
Many moderators do not have the skill set to do this themselves, so a very basic explanation is helpful.
The plan is for me to create a local Wordpress install, load the YG! as one board into the local install, export it (the board) from Wordpress (possibly using the bbPress add on if the new WP forum is going to use bbPress) to a file.
You get sent the file which you import.
Additionally I'll send you a file of users to upload first. Users have to be uploaded first.
That'll be a CSV file. There's a lot of WP addons to do CSV file user upload.
IMPORTANT: group owners need to download their members list using PGO.
Right click on group name. Choose "Get members for current group".
Wait for it to finish.
Then, right click on group name. Choose "Export members for current group".
Logan, can you email me your PGO to phpbb script?
Thanks for providing your software, forum and service.
George
Getting the data out is trivial.
Getting it in is the problem.
The phpbb migration I used only works for (IIRC) 3.0.x.
I created a new 3.0.x group and then upgraded it to 3.2.
It was a ball ache of genuinely gargantuan proportions. It took me, 6 weeks I think.
I will attach the two mods. One for user csv import and the other for messages XML import.
And the two Python progs to create the users csv and messages XML from the PG4 archive file.
You need to add users FIRST. The user add is actually not that bad. It really doesn't like apostrophes in names (like O'Brien).
if my PHP had been better I might have been able to fix that.
I have added a sample users file. Its semi-colon separated.
And a sample message XML file.
I did this three years ago so take what I say with a pinch of salt.
All of this is relatively trivial compared to the REAL task.
phpBB is sensitive as all hell to malformed HTML. Yahoo didn't give a flying **** about what the HTML looked like.
I think I wrote over 50 regexs to fix the incoming data for a group. None of which will be any use to you as the HTML issues of one group will not be the same as others.
That is the real struggle with getting YG into phpBB.
Cheers,
Wilson.
-----Original Message-----
From: Steve S [mailto:mail@mgnuts.com]
Sent: 17 October 2019 20:22
To: info@personalgroupware.com
Subject: PHPBB Migration
Hi Wilson, I'd like more info on migrating to PHPBB. I have a board already set up and I have my Yahoo Group downloaded. What is the conversion process? Also, since the PHPBB forum has been running for a couple years, the Yahoo Group will be merging into an active forum. Is there a danger or corrupting or interrupting the existing forum during the migration?
Thanks,
Steve
I am actually going to be using BBpress, but you said in other posts that you convert to phpbb first. So I could use an specific version of phpbb as long as its available. I already have used PGO to gather all my yahoo group data. Your software worked great. I figured there will be several rounds of parsing user csv and data to get it into the correct form.
George
You can go to bbpress much easier that phpbb...
Its a one step process, if I could make the program read PGO data.
It currently reads FB group data.
Wilson, I'm a bit confused and need some clarification. I am trying to go from YG to WP bbpress. I have already installed a new wordpress site and added the bbpress plugin. For my Yahoo group, I have used pg offline 4 version 4.0.907 to download all YG data. I have my members in csv form and have all my messages archived in a .pg4 file, and I exported them also to sql, and pipe deliminated text.
I have been able to successfully import all members. I am now trying to get the messages loaded.
From some of the old posts here(above), there are steps that may be eliminated due to using our newest version of PGOnline, so I am confused as to the steps I should be taking at this point.
With what I now have, how should I proceed to get my messages into bbpress from here? Which export file is the best to use.
Thanks again for providing an option for all of us in this YG regard!!
George
George,
AFAIK, there's no way to import data direct from PGO to BBpress.
I spent about 4 weeks trying. I paid for a plugin. I tried really, really hard.
And then I gave in & did what I should have done from the start and I spent two weeks writing a program to read PGO data and write to the Wordpress database directly.
It was an experience I'd rather not repeat.
Cheers,
Wilson.
Okay, got it. what would you charge to convert 24800 messages to wordpress?
"It was an experience I'd rather not repeat."
I'm not joking. I'd rather bait a crocodile with my privates.
All kidding aside, I haven't the bandwidth right now to do this personally.
I could probably find a friend to look at it for you.
Its gonna be about $250.
Cheers,
Wilson.
Wilson, Thanks for the estimate. I'm not in a tremendous rush to get the new bbpress site up even though YG goes away tomorrow. I'll probably struggle with the conversion for a bit to see if I can do it. If not, you will hear back.
George
I wish you the best of luck with that.
There may be an easier solution out there but I didn't find it.
Since PHPBB has been discussed in this thread, I'm not sure if I should start a new thread or not. If I should let me know and I will.
I saw over on GMF that there have been improvements to the PHPBB converter process. I would like to purchase the script or program (not entirely sure what form it is in currently), as I am working with quite a few groups and need to set up a read-only forum site with one per group (and there are quite a few groups - there are several of us who did what we could to preserve vintage synthesizer and music groups). Some are still active and want to move to a forum-based site, and others are simply needing an online message archive (many are Mac users too). It would be more time-efficient I feel if I could do the conversions as there are so many of them to do. Combined, including dead groups that have valuable information, I'm looking at about 350!. Please let me know if you're willing to sell me a copy of the program. I would greatly appreciate it.
Its working now.
http://www.frenchandlogan.com/phpBB3/viewforum.php?f=3&sid=4e39cdca05fe408c4a5620124d0a7b28
I do it as a service. $200 per group + $1 per 1000 records uploaded.
If you have some tremendous number to do then we can come to an arrangement.
It would probably be best to take that discussion offline.
info@personalgroupware.com
Cheers,
Wilson.
Its done....
If interested, drop me a line.
I've done some experimenting with bbPress and it is possible to do it manually, but as expected it's super time-consuming. Basically you need to use a CSV importer and split your messages into three spreadsheets/tables - one for your forum, one for topics, and one for replies. I followed this thread: https://bbpress.org/forums/topic/import-bb-data-from-csv-file-to-bbpress-forum/ (https://bbpress.org/forums/topic/import-bb-data-from-csv-file-to-bbpress-forum/). If there was a way to automate the splitting into the three different tables it would be a lot faster. But I was able to import a 50-post group into a local Wordpress install successfully, if not the cleanest result (Yahoo formatting was crap). Quotes do not work. Lots of manual cleaning up may be needed.
The problem I have is running the BBpress functions to reconnect all the parents to children etc.
Once you get to 5000+ posts the time the functions take exceed the server timeout.