Jump to content

Community

GreenLinks

Members
  • Content Count

    1,075
  • Joined

  • Days Won

    8

 Content Type 

Profiles

Downloads

IPS4 Documentation

IPS4 Providers

Release Notes

IPS4 Guides

IPS4 Developer Documentation

Invision Community Blog

Forums

Posts posted by GreenLinks


  1. It is still the case. It's not as bad, but there are still performance differences. It's going to be slower no matter what because more data is being transferred.

    Nay it is not the case , in fact many websites only choose to run on SSL because it is faster now.


  2. I disagree. SSL uses more resources, slows down page loads, and adds costs. I don't think normal page loads need to be encrypted. Only sensitive data, like passwords, should be encrypted.

    That used to be the case 4-5 years ago. I advise you to catch up with current server technology...


  3. The more i study this tool , the more i see that there was not actual much planning put on this tool.

    The perfect senario that this should be designed and function should be as following :

    CREATE DATABASE dbname CHARACTER SET utf8 COLLATE utf8_general_ci;
    

    Second step is running the conversion.

    When conversion starts CLI script should ask us which DB to read from and then again ask us which DB to write to. Lets assume DB1 and DB2 as those defined tables.

    Script should read table from DB 1 -> Converts to UTF8 -> Inserts into DB2 with utf8_general_ci

    This way when the conversion is done , boards will not have to wait for the second step current Convertor is running ( Altering collations on tables )

    This would be way safer method to follow and things would run twice faster then current method.

    It is still not to late to correct this minor planning mistake.

    I strongly suggest IPB team to make changes so even the convertor will be perfect and can be completed twice faster then it currently is.


  4. The last post from you I saw was that your site only took 35 minutes to convert with 19 mil post... that was great news... now I see this, I'll check on your ticket and I'm sure we will address any issues though and do the best we can to adapt for larger sites.

    Yes , however that was when we first noticed the time out issues.

    Another thing i noticed is those timeouts specifically occur when you use 4-byte UTF-8 Encoding.

    On my mac pro total time to complete the conversion is around 5 hours vs 8 hours on test server but the difference was Use 4-byte UTF-8 Encoding

    Merts-Mac-Pro:GcForum lizardking$ php utf8convert/cli.php 
    ------------------------------------------
    Welcome to the IPS UTF8 Conversion utility
    ------------------------------------------
    
    303 table(s) are not UTF-8 and need converting.
    
    Use 4-byte UTF-8 Encoding (utf8mb4)?
    Some non-common symbols (such as historical scripts, music symbols and Emoji) require more space in the database to be stored. If you choose 'no', users will not be able to use these symbols on your site. If 'yes', these characters will be able to be used, but the database will use more disk space.
    [y] Enter 'y' for Yes
    [n] Enter 'n' for No
    n
    This utility will convert all the tables in this database to UTF-8.
    The converted data is inserted into new tables prefixed with x_utf_ and the original data kept.
    Press enter to continue
    
    Processing:                                                                             100%   
    Conversion has completed in 2 hour(s), 24 minute(s) and 47 seconds.
    [f] Enter 'f' to finish
    [x] Enter 'x' to reset and start the conversion again.
    f
    You can now test the conversion by:
    
    Editing conf_global.php to add $INFO['sql_charset'] = 'UTF8';
    Change $INFO['sql_tbl_prefix'] to: $INFO['sql_tbl_prefix'] = 'x_utf_';
    
    If you are happy with the conversion, you can have it permanently rename the tables so that you can restore your original 'sql_table_prefix' and delete the original tables.
    [y] Enter 'y' to rename your tables
    [x] Enter 'x' to reset and start the conversion again.
    y
    Renaming tables, this may take a short while.
    Checking and fixing collations.
    You can now complete the conversion by editing conf_global.php to add $INFO['sql_charset'] = 'UTF8'; and to change $INFO['sql_tbl_prefix'] to:
    $INFO['sql_tbl_prefix'] = '';
    

    I still have all the orig named tables which is clearly a sign that the convertor only received minor corrections.


  5. Sure lets report some bugs which are mainly highlighted in the ticket already :

    1. Timeout after huge table conversions
    2. Script fails to remove orig. named tables which doubles database sizes ( requires manual work , unacceptable )
    3. Indexes are deleted during the conversion

    These are explained in the ticket ( expect indexes ) but we don't know if they are fixed or not . I did a final test after yesterdays upgrade ( all bugs unfortunately still exist )

    Each test takes around 8 hours so if you are expecting us to go on with these tests , then you shall first act and implement improvements that will ease our job.


  6. If the team doesn't make necessary changes to assist big board owners within their tests , then highly possible we will stop reporting the bugs Aiwa.

    This conversion process needs to be flawless, thats why it shall include the most effective test practises which lays on having all changes on separate databases.

    I am sorry but i seriously don't see a reason how you can arguing against this.

    Math is extremely simple here and the outcome is again extremely simple.

    If anything goes wrong :

    If conversion done on a separate database :

    Dropping the secondary database - creating a new database takes : 2 min max

    If conversion done on same database :

    Multiple choices

    1. Clean the current database -> 60-90 min manual work
    2. Drop your test database, create a new database and first import your backup -> At least 60-70 minute wait time

  7. We talking about the converter that converts you over from another software, or the UTF8 converter? I'm assuming the utf8 converter, as I saw you were testing it the other day....

    The utf8 converter creates NEW tables in your database so it's not messing with your live site. Then you simply update your conf_global sql prefix to test the conversion... Once it's good, you switch it back and finalize the utf8 conversion. If the conversion doesn't go well, you only need to drop the NEW tables, and ensure your conf_global sql prefix is pointing to your original tables, not the whole database...

    Why would you have import your database?

    If anything goes wrong within the convertor example ( on our last test conversion we found some table renaming missing ) , you basically need to start from scratch or manually have to rename all those tables. Which is another major time consuming thing

    UTF8 Convertor should work as simple as possible and should not interfere with the main database. That shall always be the principle.


  8. Hello ,

    Convertor may work for small websites in current state but for big boards it is not feasible.
    Current format risks too long down times for any big board. The tables are being forced to be created on same database. So if anything goes wrong big boards need to drop their database , reimport the database and start from scratch.

    Exporting a database takes around : 15 min

    Importing a database takes around : 1 hour

    Conversion is completed around : 2 hour 30 minute

    Collation change is taking around : 5 hours to compete

    You need to make sure to eliminate the need of exporting and importing the databases so we will not have to be forced to take down our boards longer then it is needed.

    If we assume upgrade is going to take around 2 hour ( table changes etc. ) IPB teams main focus should be on making sure no additional senario's can cause downtimes.

    So allowing a separate database should be the first target atm...


  9. Your suggestion makes sense from a point. Why reinvent the wheel .

    However there are major problems with that approach for many different cases.

    For example look at Magento , they need to use a heavily customised Zend version and have to invest on Zend development also.

    Your suggestions may sound perfect choice for your own usage but from my stand point suggested frameworks by you are really not great choices.

    They all have major flaws in my opinion and i personally stay away from any script that is build on those frameworks.

    As you can see your argument will not even hold against another regular customer.

    The team is responsible with developing IPB. They were hired because IPB management felt they are good developers. So instead of trying to teach them their job , why not await what the team will bring to the table and then voice your opinion ?


  10. We will like to see a better UI that clearly displays the most recent activity on gallery. Currently it is kind of major problem to find out updated images imo.

    Also we will like to see major improvements on Image Upload process . It is kind of problematic currently.

    Photo Tagging is another critical feature we are looking for.

    Hopefully Gallery will also have way better video features.

    Also, why not make it possible to sell photos?

    Like: Allow to add additional image who does get sold(so u can upload 1 file wich is downloaded by the one who bought it)(and remove the "large" view size image)
    and make it a permission option. (Add a "Terms of Use" page for the images that get sold)

    within acc, only allow PayPal (add the member email in user panel under gallery)

    Also add commission to the board - if i buy a image for 20$ the admin(or something) gets(choose in admin) 20% - 70% of the amount.

    This is a great idea however i would love to see full support to all installed Payment Gateways and an ability to mass pay image owners via paypal or other channels.


  11. I'll overlook the fact that you quoted a post I made 2.5 years ago lol - things do change as time goes on and software progresses and technology changes and so on.

    As Mark mentioned, a wait_timeout value of 28800 should not be necessary. A query in our software (or any really) should not take 28800 seconds to execute and return. I would generally aim for 60 seconds and don't typically see any issues with that, but YMMV.

    Anyways, you are seeing connection timeouts in your logs you say? What is the general symptom that you are seeing? Queries that are sent to the server, but the connection is closed before the response is received back by the software?

    The topic was bumped before me Brandon. I read it and responded , we went over the issue on an old support ticket. The connection between DB , Web server is extremely fast. They are connected through a direct 10GB line and the connection is through MySQL socket for performance improvements. However if you prefer , i can request our Sysadmin to supply detailed information via a support ticket so we can both study the possible causes.


  12. Could you elaborate on what the issue you're experiencing is and how you think we could improve? I can't think of any particular reason why IP.Board would require any particular wait_timeout.

    Naturally, our support agents will recommend the default values, but yes, you're absolutely right that, if you know what you're doing, reducing that shouldn't be an issue.

    Mark , i have daily around 10+ MB connection timeout queries logged in SQL error logs. We learn to live with this , if we reduce values to acceptable levels where we run vBulletin without any issue, the connection timeout errors are reaching to 100MB+ logs and basically IPB becomes unresponsive for a time.

    The main reason is the number of queries every page requires by IPB. Again hopefully all these issues will be resolved with IPB 4. However as of now , IPB has issues when you average 10+ k online users.

    We hope the suggestion of moving active / inactive users to separate tables and major improvements to topic archiving system. Current format is not usable for us.

    P.S : I never understand users whom constantly suggest pruning users. A system like IPB should run without the need of pruning any users.


  13. There are many large forums using IP.Board. Most have no issues with scalability, or the issues they face require server-side provisioning and not necessarily software changes.

    We are constantly working to improve efficiency in IP.Board, from little changes to big changes. Not every issue is as black and white as it may seem on the surface.

    I do not think you would have any problems with using IP.Board for a "huge" website, provided you have the server resources to back the software up. Whether you run IP.Board or any other software, if your site is huge, you need a good enough server to handle the activity.

    Sorry but not fully true Brandon :)

    And i fully support all the other big board owners that wanted to state this absolute truth to you. There are fundemental errors that causes the issues.

    Biggest issue for me is the requirement for high MySQL wait time. Your support engineers constantly suggest default value which is 28800 . It is an unacceptable level of setting for any big board. This is something hopefully will be nailed with IPB 4.

×
×
  • Create New...