Moving from Yahoo Groups to bbPress using PGO

[Josh]

First of all, thanks for providing an excellent piece of software for free. PGO is exactly what I needed for at least part of what I'm trying to do.

I realize this question isn't completely relevant to PGO, but I saw that you provided help with similar problems so I'm hoping you'll be able to point me in the right direction here.

I'm trying to move a Yahoo group to bbPress. I have the messages and topics downloaded via PGO, which is the first step. bbPress includes an importer than can import a database from basically every other popular forum software, but not directly from Yahoo groups. So, I'm guessing the best method will be to convert the Yahoo data into a format that can be imported into one of the more popular forum softwares, and then import that database into bbPress.

The issue I'm having is how to convert the data I have into a format that can then be imported into another forum software. I basically have two related questions here:

1. Can I convert from .db3 into a format that can then be imported into something like phpBB or vBulletin?

2. If so, what format would that be, and what importers are available? I'm having trouble finding an importer for any of the major forums that will allow me to import from a raw format like .csv, or whatever I could get the PGO data converted to. It doesn't matter what forum software I use, since basically all of them can be imported through bbPress. It's just a matter of finding one that has an importer that will allow me to import the PGO data.

Hopefully that makes sense. Thanks in advance for any help you can provide.

[Wilson Logan]

I don't think there's going to be any other method than doing a bit of work to format the PGO data into a format that your importer of choice can consume.

It shouldn't be that hard. The basic data that any forum will require is

Author
Date
Time
Message subject
Message Text


Looking at your specific problem... bbPress doesn't seem to want a flat file input. It wants another forum format to leach from.

That means that if you want bbPress you will have to import the PGO data first to a forum bbPress can read.

I would suggest phpBB. I have some scripts to convert the old PGO .mdb archive to phpBB 3 (somewhere).

You'd just need to change the script that reads the .mdb database and change it from .mdb to .db3


Cheers,

Wilson.

[Josh]

Thanks for the advice. Would you be able to provide that script? That would help a lot. I'm assuming it would preserve the data you mentioned, so it would just be a matter of converting from .db3 -> phpBB database format (whatever that is) -> bbPress? As you said, if I could get the data into a database that functioned with phpBB3 then importing that using bbPress should be no problem.

[Wilson Logan]

I'll have to find those scripts. The last time I saw them was > 3 years ago.

They're fairly simple and there is a good set of instructions that go with them.

Anyway, give me a couple of days to find them...

Cheers,

Wilson.

[Josh]

No problem, I really appreciate the help.

[Wilson Logan]

OK, I have the scripts.

I will contact you via email regarding their use.

Cheers,

Wilson.

[Wilson Logan]

Update....

I now have a program to convert the PGO groups database to a format that can be consumed by the phpbb Message & User import mods.

Additionally, there is a plugin for bbPress to import phpBB data which I have tested and can verify works.

Cheers,

Wilson.

[casstk]

Would you be willing to license the PGO->phbb program? I know you offer a custom import service but for smaller yahoo groups the $100 price may be too much.

[Wilson Logan]

Sure I'll sell you cheap access to the phpBB converter but be aware, it'll still be like pulling teeth:

You will require:

1) A copy of XAMMP to create and run local copies of whatever forum software you decide to run + MySQL + Apache.

2) A working knowledge of PHP (assuming phpBB or Wordpress)

3) Good / very good  knowledge of MySQL query language

4) A working knowledge of Python or C++.

5) Hosting for your forum.

[casstk]

I am writing up info for fellow YG moderators.  It will help to tell them what your fee covers because I agree, it is worth every penny.

Ex:
"You can move your Group to WordPress if
1. You have a self-installed version of Wordpress on your own domain name
2. Know how to install plugins (ex: BBPress)
3. Have $100 to pay Wilson to convert your YG into a format that can be imported (for the first xx thousand messages, the cost for each xx thousand next messages is.....?????

For that fee all you have to do is ...???  give Wilson admin access to the group? or does he need to only be a member?  give him admin access to your WordPress install?  Does the moderator  have to do anything else to get it up and running? Configure???

Many moderators do not have the skill set to do this themselves, so a very basic explanation is helpful.

[Wilson Logan]

The plan is for me to create a local Wordpress install,  load the YG! as one board into the local install, export it (the board) from Wordpress  (possibly using the bbPress add on if the new WP forum is going to use bbPress) to a file.

You get sent the file which you import.

Additionally I'll send you a file of users to upload first.  Users have to be uploaded first.

That'll be a CSV file. There's a lot of WP addons to do CSV file user upload.

IMPORTANT:  group owners need to download their members list using PGO.

Right click on group name. Choose "Get members for current group".

Wait for it to finish.

Then, right click on group name. Choose "Export members for current group".

[glcost]

Logan,  can you email me your PGO to phpbb script?

Thanks for providing your software, forum and service.

George

[Wilson Logan]

Getting the data out is trivial.

Getting it in is the problem.


The phpbb migration I used only works for (IIRC) 3.0.x.

I created a new 3.0.x group and then upgraded it to 3.2.

It was a ball ache of genuinely gargantuan proportions. It took me, 6 weeks I think.

I will attach the two  mods.   One for user csv import and the other for messages XML import.

And the two Python progs to create the users csv and messages XML from the PG4 archive file.

You need to add users FIRST. The user add is actually not that bad.  It really doesn't like apostrophes in names (like O'Brien).

if my PHP had been better I might have been able to fix that.

I have added a sample users file. Its semi-colon separated.

And a sample message XML file.

I did this three years ago so take what I say with a pinch of salt.



All of this is relatively trivial compared to the REAL task. 

phpBB is sensitive as all hell to malformed HTML.  Yahoo didn't give a flying **** about what the HTML looked like.

I think I wrote over 50 regexs to fix the incoming data for a group.  None of which will be any use to you as the HTML issues of one group will not be the same as others.

That is the real struggle with getting YG into phpBB.


Cheers,

Wilson.


-----Original Message-----
From: Steve S [mailto:mail@mgnuts.com]
Sent: 17 October 2019 20:22
To: info@personalgroupware.com
Subject: PHPBB Migration

Hi Wilson, I'd like more info on migrating to PHPBB.  I have a board already set up and I have my Yahoo Group downloaded.  What is the conversion process?  Also, since the PHPBB forum has been running for a couple years, the Yahoo Group will be merging into an active forum.  Is there a danger or corrupting or interrupting the existing forum during the migration?

Thanks,

Steve

[glcost]

I am actually going to be using BBpress, but you said in other posts that you convert to phpbb first. So I could use an specific version of phpbb as long as its available. I already have used PGO to gather all my yahoo group data. Your software worked great. I figured there will be several rounds of parsing user csv and data to get it into the correct form.

George

[Wilson Logan]

You can go to bbpress much easier that phpbb...

Its a one step process, if I could make the program read PGO data.

It currently reads FB group data.