List:General Discussion« Previous MessageNext Message »
From:Rhino Date:August 19 2004 4:39pm
Subject:Re: 1 day 28 min insert
View as plain text  
----- Original Message ----- 
From: "matt ryan" <matt_lists@stripped>
To: <mysql@stripped>
Sent: Thursday, August 19, 2004 12:06 PM
Subject: 1 day 28 min insert


> I think oracle parallel query is calling me
>
> 110,832,565 stat records
>
> 77,269,086 on weekly update, I get small daily files, but daily sql's
> dont work very well, and miss records, in this case it missed 563 records.
>
> mysql> update stat_in set ctasc='321ST';
> Query OK, 77269086 rows affected (24 min 17.60 sec)
> Rows matched: 77269086  Changed: 77269086  Warnings: 0
>
> mysql> insert ignore into 321st_stat select * from stat_in;
> Query OK, 563 rows affected (1 day 28 min 35.95 sec)
> Records: 77269086  Duplicates: 77268523  Warnings: 0
>
> I just cant deal with speeds this slow, an insert onto a table with a
> primary key that tosses out almost all records shouldnt take this long to
do
>
Your post is very cryptic and I really don't understand just what you want.

Do you just want people to agree that 24+ hours is too long for an insert to
take place? If so, I would agree that 24 hours is way too long to insert a
*single* record. However, I've never tried inserting 77 million records so
I'm not sure if 24 hours is unreasonable for that. Maybe others who have
more experience with large databases can comment on that.

Or are you asking a question? If so, what exactly *is* your question? Are
you worried about why some records were missed or why the insert took so
long or both?

If you're trying to get some help with a problem, you'll have to give us
more information. At the very least, we need to know something about your
environment, like which version of MySQL you are using. We're likely to need
to know more as we start to understand the question, such as your table
definition(s), and we may need to see the code that is doing the inserts to
see if we can help you optimize it. We may even need information about your
hardware if this is a performance question to see whether the times you are
getting are reasonable for your hardware.

Rhino

Thread
1 day 28 min insertmatt ryan19 Aug
  • Re: 1 day 28 min insertAndrew Pattison19 Aug
    • Re: 1 day 28 min insertmatt ryan19 Aug
      • Re: 1 day 28 min insertSGreen19 Aug
    • Re: 1 day 28 min insertSGreen19 Aug
  • Re: 1 day 28 min insertDan Nelson19 Aug
    • Re: 1 day 28 min insertmatt ryan19 Aug
      • Re: 1 day 28 min insertDan Nelson19 Aug
        • RE: 1 day 28 min insertBoyd E. Hemphill19 Aug
          • Re: 1 day 28 min insertmatt ryan19 Aug
            • Re: 1 day 28 min insertRemigiusz SokoĊ‚owski20 Aug
              • RE: 1 day 28 min insertDonny Simonton20 Aug
  • Re: 1 day 28 min insertRhino19 Aug
  • Re: 1 day 28 min insertMikhail Entaltsev20 Aug
    • Re: 1 day 28 min insertmatt ryan27 Aug
  • Re: 1 day 28 min insertMikhail Entaltsev27 Aug
    • Re: 1 day 28 min insertmatt ryan27 Aug
      • Re: 1 day 28 min insertSGreen27 Aug
        • Re: 1 day 28 min insertMikhail Entaltsev27 Aug
          • Re: 1 day 28 min insertSGreen27 Aug
            • Re: 1 day 28 min insertMikhail Entaltsev27 Aug
          • Re: 1 day 28 min insertmatt ryan1 Sep
  • Re: 1 day 28 min insertMikhail Entaltsev27 Aug